[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:24:17 dist-test-slave-kr9h krb5kdc[7078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:24:17 dist-test-slave-kr9h krb5kdc[7078](info): set up 2 sockets
Jun 19 12:24:17 dist-test-slave-kr9h krb5kdc[7078](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260619 12:24:18.270157 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:24:18 dist-test-slave-kr9h krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:24:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:24:18.397127 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:43389
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:43389
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:18.508114 7091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:18.508459 7091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:18.508558 7091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:18.512764 7091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:18.512869 7091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:18.512907 7091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:18.512945 7091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:18.512979 7091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:18.518728 7091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:43389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:43389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:18.519971 7091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:18.521153 7091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:18.530663 7096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:18.530683 7097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:18.530750 7099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:18.530728 7091 server_base.cc:1034] running on GCE node
I20260619 12:24:18.531534 7091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:18.532706 7091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:18.533890 7091 hybrid_clock.cc:648] HybridClock initialized: now 1781871858533768 us; error 136 us; skew 500 ppm
Jun 19 12:24:18 dist-test-slave-kr9h krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871858, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:18.543093 7091 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:24:18.544162 7091 webserver.cc:460] Webserver started at http://127.6.231.254:38639/ using document root <none> and password file <none>
I20260619 12:24:18.544725 7091 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:18.544790 7091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:18.544961 7091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:18.547147 7091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "f87155fc69f740b696bac512382384e2"
format_stamp: "Formatted at 2026-06-19 12:24:18 on dist-test-slave-kr9h"
server_key: "3c5026cc7bbd4840d2adf9e5d33ba481"
server_key_iv: "816b4e9b620b597a54b66d1c30b02f99"
server_key_version: "encryptionkey@0"
I20260619 12:24:18.547688 7091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "f87155fc69f740b696bac512382384e2"
format_stamp: "Formatted at 2026-06-19 12:24:18 on dist-test-slave-kr9h"
server_key: "3c5026cc7bbd4840d2adf9e5d33ba481"
server_key_iv: "816b4e9b620b597a54b66d1c30b02f99"
server_key_version: "encryptionkey@0"
I20260619 12:24:18.551587 7091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:24:18.554283 7106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:18.555821 7091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:18.555977 7091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "f87155fc69f740b696bac512382384e2"
format_stamp: "Formatted at 2026-06-19 12:24:18 on dist-test-slave-kr9h"
server_key: "3c5026cc7bbd4840d2adf9e5d33ba481"
server_key_iv: "816b4e9b620b597a54b66d1c30b02f99"
server_key_version: "encryptionkey@0"
I20260619 12:24:18.556151 7091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:18.569609 7091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:18.573525 7091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:18.573800 7091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:18.581880 7091 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:43389
I20260619 12:24:18.581952 7158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:43389 every 8 connection(s)
I20260619 12:24:18.582988 7091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:18.584059 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 7091
I20260619 12:24:18.584261 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:18.585783 7071 external_mini_cluster.cc:1428] Setting key 167a0ce65197626af887d3cff9118eab
I20260619 12:24:18.586258 7159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:18.592494 7159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2: Bootstrap starting.
I20260619 12:24:18.595136 7159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:18.595924 7159 log.cc:824] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:18.598315 7159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2: No bootstrap required, opened a new log
I20260619 12:24:18.602352 7159 raft_consensus.cc:348] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f87155fc69f740b696bac512382384e2" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43389 } }
I20260619 12:24:18.602613 7159 raft_consensus.cc:374] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:18.602707 7159 raft_consensus.cc:729] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f87155fc69f740b696bac512382384e2, State: Initialized, Role: FOLLOWER
I20260619 12:24:18.603226 7159 consensus_queue.cc:260] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f87155fc69f740b696bac512382384e2" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43389 } }
I20260619 12:24:18.603430 7159 raft_consensus.cc:388] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:18.603514 7159 raft_consensus.cc:482] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:18.603641 7159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:18.605180 7159 raft_consensus.cc:504] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f87155fc69f740b696bac512382384e2" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43389 } }
Jun 19 12:24:18 dist-test-slave-kr9h krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:18.605615 7159 leader_election.cc:302] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f87155fc69f740b696bac512382384e2; no voters:
I20260619 12:24:18.606010 7159 leader_election.cc:288] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:18.606086 7164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:18.606292 7164 raft_consensus.cc:686] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [term 1 LEADER]: Becoming Leader. State: Replica: f87155fc69f740b696bac512382384e2, State: Running, Role: LEADER
I20260619 12:24:18.606602 7164 consensus_queue.cc:237] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f87155fc69f740b696bac512382384e2" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43389 } }
I20260619 12:24:18.607376 7159 sys_catalog.cc:564] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:18.608297 7165 sys_catalog.cc:455] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f87155fc69f740b696bac512382384e2. Latest consensus state: current_term: 1 leader_uuid: "f87155fc69f740b696bac512382384e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f87155fc69f740b696bac512382384e2" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43389 } } }
I20260619 12:24:18.608451 7165 sys_catalog.cc:458] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:18.608728 7166 sys_catalog.cc:455] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f87155fc69f740b696bac512382384e2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f87155fc69f740b696bac512382384e2" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43389 } } }
I20260619 12:24:18.608824 7166 sys_catalog.cc:458] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2 [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:18.609095 7173 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:18.612638 7173 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:18.613680 7161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:18.589118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60018 (local address 127.6.231.254:43389)
0619 12:24:18.589653 (+ 535us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:18.589674 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:18.590851 (+ 1177us) server_negotiation.cc:407] Connection header received
0619 12:24:18.594322 (+ 3471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:18.594343 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:18.594754 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:18.595114 (+ 360us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:18.597519 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:18.599020 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:18.600395 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:18.601213 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:18.601386 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:18.606708 (+ 5322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:18.606717 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:18.606734 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:18.606770 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:18.609041 (+ 2271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:18.609951 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:18.609957 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:18.609963 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:18.610117 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:18.610588 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:18.610592 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:18.610593 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:18.611153 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:18.611375 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:18.612109 (+ 734us) server_negotiation.cc:299] Negotiation successful
0619 12:24:18.612371 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":159,"threads_started":1}
I20260619 12:24:18.618911 7173 catalog_manager.cc:1347] Generated new cluster ID: 9ba5c15d88334014a99d94215b72cc69
I20260619 12:24:18.619006 7173 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:18.635320 7173 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:18.637387 7173 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:18.648082 7173 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f87155fc69f740b696bac512382384e2: Generated new TSK 0
I20260619 12:24:18.648893 7173 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:24:18.682899 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:43389
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:18.799312 7185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:18.799571 7185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:18.799615 7185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:18.803637 7185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:18.803702 7185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:18.803764 7185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:24:18.809458 7185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:43389
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:18.810585 7185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:18.811574 7185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:18.821173 7190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:18.821174 7193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:18.821174 7191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:18.821530 7185 server_base.cc:1034] running on GCE node
I20260619 12:24:18.822037 7185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:18.822765 7185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:18.823985 7185 hybrid_clock.cc:648] HybridClock initialized: now 1781871858823970 us; error 32 us; skew 500 ppm
Jun 19 12:24:18 dist-test-slave-kr9h krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871858, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:18.827198 7185 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:24:18.828027 7185 webserver.cc:460] Webserver started at http://127.6.231.193:45677/ using document root <none> and password file <none>
I20260619 12:24:18.828547 7185 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:18.828604 7185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:18.828773 7185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:18.830935 7185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "248547ce4e5b40258ae6cba8e6786e4d"
format_stamp: "Formatted at 2026-06-19 12:24:18 on dist-test-slave-kr9h"
server_key: "262a74105ffabf865f548ffcba056975"
server_key_iv: "209501078cda61ce5553279f1fcca838"
server_key_version: "encryptionkey@0"
I20260619 12:24:18.831439 7185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "248547ce4e5b40258ae6cba8e6786e4d"
format_stamp: "Formatted at 2026-06-19 12:24:18 on dist-test-slave-kr9h"
server_key: "262a74105ffabf865f548ffcba056975"
server_key_iv: "209501078cda61ce5553279f1fcca838"
server_key_version: "encryptionkey@0"
I20260619 12:24:18.835150 7185 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260619 12:24:18.837653 7200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:18.838974 7185 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 12:24:18.839109 7185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "248547ce4e5b40258ae6cba8e6786e4d"
format_stamp: "Formatted at 2026-06-19 12:24:18 on dist-test-slave-kr9h"
server_key: "262a74105ffabf865f548ffcba056975"
server_key_iv: "209501078cda61ce5553279f1fcca838"
server_key_version: "encryptionkey@0"
I20260619 12:24:18.839224 7185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:18.858203 7185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:18.861784 7185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:18.862001 7185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:18.862623 7185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:18.863511 7185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:18.863559 7185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:18.863623 7185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:18.863655 7185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:18.872905 7185 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:36289
I20260619 12:24:18.872972 7313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:36289 every 8 connection(s)
I20260619 12:24:18.873992 7185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:24:18.880260 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 7185
I20260619 12:24:18.880369 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:24:18.880651 7071 external_mini_cluster.cc:1428] Setting key 0c005e3a75d095ac757ea5d6902f435f
Jun 19 12:24:18 dist-test-slave-kr9h krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871858, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:18.889717 7161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:18.876009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:34843 (local address 127.6.231.254:43389)
0619 12:24:18.876179 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:18.876183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:18.877057 (+ 874us) server_negotiation.cc:407] Connection header received
0619 12:24:18.878578 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:18.878581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:18.878637 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:18.878751 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:18.880000 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:18.880873 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:18.882551 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:18.883009 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:18.883133 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:18.886002 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:18.886005 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:18.886008 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:18.886031 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:18.887585 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:18.888137 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:18.888145 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:18.888147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:18.888219 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:18.888566 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:18.888569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:18.888570 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:18.888759 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:18.888863 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:18.889374 (+ 511us) server_negotiation.cc:299] Negotiation successful
0619 12:24:18.889532 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
I20260619 12:24:18.890565 7316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:18.876414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:43389 (local address 127.6.231.193:34843)
0619 12:24:18.876887 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:18.876934 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:18.878353 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:18.878935 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:18.878943 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:18.879364 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:18.879850 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:18.879863 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:18.881007 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:18.881010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:18.882431 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:18.882441 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:18.883164 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:18.883167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:18.883341 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:18.884023 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:18.884049 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:18.885826 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:18.887737 (+ 1911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:18.887743 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:18.887760 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:18.888023 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:18.888317 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:18.888321 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:18.888323 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:18.888466 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:18.888872 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:18.888881 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:18.889129 (+ 248us) client_negotiation.cc:769] Sending connection context
0619 12:24:18.889323 (+ 194us) client_negotiation.cc:240] Negotiation successful
0619 12:24:18.889675 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":105,"threads_started":1}
I20260619 12:24:18.891855 7314 heartbeater.cc:347] Connected to a master server at 127.6.231.254:43389
I20260619 12:24:18.892107 7314 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:18.892707 7314 heartbeater.cc:511] Master 127.6.231.254:43389 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:24:18.894614 7123 ts_manager.cc:194] Registered new tserver with Master: 248547ce4e5b40258ae6cba8e6786e4d (127.6.231.193:36289)
I20260619 12:24:18.895967 7123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:34843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:24:18.909008 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:43389
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:19.023686 7319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:19.024011 7319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:19.024122 7319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:19.028898 7319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:19.028972 7319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:19.029074 7319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:24:19.035166 7319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:43389
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:19.036368 7319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:19.037505 7319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:19.047528 7327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:19.047657 7325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:19.047574 7324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:19.047976 7319 server_base.cc:1034] running on GCE node
I20260619 12:24:19.048374 7319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:19.049064 7319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:19.050257 7319 hybrid_clock.cc:648] HybridClock initialized: now 1781871859050236 us; error 31 us; skew 500 ppm
Jun 19 12:24:19 dist-test-slave-kr9h krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871859, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:19.053753 7319 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:24:19.054666 7319 webserver.cc:460] Webserver started at http://127.6.231.194:40331/ using document root <none> and password file <none>
I20260619 12:24:19.055275 7319 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:19.055372 7319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:19.055609 7319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:19.057822 7319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "dcf035631daf457087a19a0ab44a79fa"
format_stamp: "Formatted at 2026-06-19 12:24:19 on dist-test-slave-kr9h"
server_key: "2ad5e6693db356b49c680ea362e638e0"
server_key_iv: "4a42df3bbdf38096f8999d2e8d77832e"
server_key_version: "encryptionkey@0"
I20260619 12:24:19.058432 7319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "dcf035631daf457087a19a0ab44a79fa"
format_stamp: "Formatted at 2026-06-19 12:24:19 on dist-test-slave-kr9h"
server_key: "2ad5e6693db356b49c680ea362e638e0"
server_key_iv: "4a42df3bbdf38096f8999d2e8d77832e"
server_key_version: "encryptionkey@0"
I20260619 12:24:19.062790 7319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:24:19.065424 7334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:19.066732 7319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:24:19.066877 7319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "dcf035631daf457087a19a0ab44a79fa"
format_stamp: "Formatted at 2026-06-19 12:24:19 on dist-test-slave-kr9h"
server_key: "2ad5e6693db356b49c680ea362e638e0"
server_key_iv: "4a42df3bbdf38096f8999d2e8d77832e"
server_key_version: "encryptionkey@0"
I20260619 12:24:19.067051 7319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:19.080327 7319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:19.083703 7319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:19.083923 7319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:19.084532 7319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:19.085367 7319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:19.085444 7319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:19.085569 7319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:19.085619 7319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:19.095772 7319 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:43637
I20260619 12:24:19.095790 7447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:43637 every 8 connection(s)
I20260619 12:24:19.096798 7319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:24:19.105966 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 7319
I20260619 12:24:19.106109 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:24:19.106418 7071 external_mini_cluster.cc:1428] Setting key 00ffcc4317997c9eb642248948cc12ca
Jun 19 12:24:19 dist-test-slave-kr9h krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871859, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:19.112093 7161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:19.098807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:35395 (local address 127.6.231.254:43389)
0619 12:24:19.098954 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:19.098958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:19.099844 (+ 886us) server_negotiation.cc:407] Connection header received
0619 12:24:19.101218 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:19.101221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:19.101278 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:19.101390 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:19.102635 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:19.103160 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:19.104336 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:19.104553 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:19.104668 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:19.107426 (+ 2758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:19.107430 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:19.107432 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:19.107456 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:19.109904 (+ 2448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:19.110424 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:19.110430 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:19.110432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:19.110528 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:19.110963 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:19.110969 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:19.110975 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:19.111157 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:19.111304 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:19.111796 (+ 492us) server_negotiation.cc:299] Negotiation successful
0619 12:24:19.111931 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 12:24:19.112974 7450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:19.099217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:43389 (local address 127.6.231.194:35395)
0619 12:24:19.099688 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:19.099729 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:19.100996 (+ 1267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:19.101615 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:19.101623 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:19.101978 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:19.102480 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:19.102494 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:19.103287 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:19.103290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:19.104192 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:19.104201 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:19.104705 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:19.104709 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:19.104882 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:19.105537 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:19.105553 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:19.107264 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:19.110016 (+ 2752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:19.110022 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:19.110035 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:19.110307 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:19.110636 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:19.110643 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:19.110644 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:19.110783 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:19.111298 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:19.111304 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:19.111561 (+ 257us) client_negotiation.cc:769] Sending connection context
0619 12:24:19.111758 (+ 197us) client_negotiation.cc:240] Negotiation successful
0619 12:24:19.112102 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":100,"threads_started":1}
I20260619 12:24:19.114334 7448 heartbeater.cc:347] Connected to a master server at 127.6.231.254:43389
I20260619 12:24:19.114580 7448 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:19.115070 7448 heartbeater.cc:511] Master 127.6.231.254:43389 requested a full tablet report, sending...
I20260619 12:24:19.116214 7123 ts_manager.cc:194] Registered new tserver with Master: dcf035631daf457087a19a0ab44a79fa (127.6.231.194:43637)
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
I20260619 12:24:19.117033 7123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:35395
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:24:19.136472 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:43389
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:19.252907 7453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:19.253185 7453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:19.253262 7453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:19.257349 7453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:19.257414 7453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:19.257572 7453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:24:19.263228 7453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:43389
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:19.264922 7453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:19.266019 7453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:19.275861 7461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:19.275861 7459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:19.275914 7453 server_base.cc:1034] running on GCE node
W20260619 12:24:19.275861 7458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:19.276579 7453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:19.277246 7453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:19.278450 7453 hybrid_clock.cc:648] HybridClock initialized: now 1781871859278446 us; error 52 us; skew 500 ppm
Jun 19 12:24:19 dist-test-slave-kr9h krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871859, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:19.281834 7453 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:24:19.282811 7453 webserver.cc:460] Webserver started at http://127.6.231.195:40881/ using document root <none> and password file <none>
I20260619 12:24:19.283392 7453 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:19.283490 7453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:19.283725 7453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:19.286247 7453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "ea9a16ca51a74393b01af8c7136171b7"
format_stamp: "Formatted at 2026-06-19 12:24:19 on dist-test-slave-kr9h"
server_key: "c408102bbc6e6d0b4b5f5c0d4872ca2f"
server_key_iv: "e3f83c88ee0ddab00ce919be4a7995c1"
server_key_version: "encryptionkey@0"
I20260619 12:24:19.286839 7453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "ea9a16ca51a74393b01af8c7136171b7"
format_stamp: "Formatted at 2026-06-19 12:24:19 on dist-test-slave-kr9h"
server_key: "c408102bbc6e6d0b4b5f5c0d4872ca2f"
server_key_iv: "e3f83c88ee0ddab00ce919be4a7995c1"
server_key_version: "encryptionkey@0"
I20260619 12:24:19.290663 7453 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260619 12:24:19.293128 7468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:19.294504 7453 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:24:19.294636 7453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "ea9a16ca51a74393b01af8c7136171b7"
format_stamp: "Formatted at 2026-06-19 12:24:19 on dist-test-slave-kr9h"
server_key: "c408102bbc6e6d0b4b5f5c0d4872ca2f"
server_key_iv: "e3f83c88ee0ddab00ce919be4a7995c1"
server_key_version: "encryptionkey@0"
I20260619 12:24:19.294744 7453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:19.309409 7453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:19.313084 7453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:19.313294 7453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:19.313937 7453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:19.314790 7453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:19.314834 7453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:19.314898 7453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:19.314929 7453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:19.325024 7453 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:46249
I20260619 12:24:19.325069 7581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:46249 every 8 connection(s)
I20260619 12:24:19.325978 7453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:24:19.333178 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 7453
I20260619 12:24:19.333420 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:24:19.333847 7071 external_mini_cluster.cc:1428] Setting key ee223a0196444721617576276258e005
Jun 19 12:24:19 dist-test-slave-kr9h krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871859, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:19.340984 7161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:19.327837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:34987 (local address 127.6.231.254:43389)
0619 12:24:19.327979 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:19.327983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:19.328820 (+ 837us) server_negotiation.cc:407] Connection header received
0619 12:24:19.330313 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:19.330316 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:19.330362 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:19.330443 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:19.331740 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:19.332308 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:19.333677 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:19.334082 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:19.334214 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:19.337354 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:19.337358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:19.337360 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:19.337383 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:19.338979 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:19.339532 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:19.339534 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:19.339536 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:19.339610 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:19.339936 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:19.339938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:19.339940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:19.340174 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:19.340253 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:19.340754 (+ 501us) server_negotiation.cc:299] Negotiation successful
0619 12:24:19.340859 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 12:24:19.341958 7584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:19.328245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:43389 (local address 127.6.231.195:34987)
0619 12:24:19.328669 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:19.328710 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:19.330086 (+ 1376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:19.330682 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:19.330690 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:19.331076 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:19.331598 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:19.331613 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:19.332426 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:19.332429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:19.333548 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:19.333562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:19.334270 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:19.334274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:19.334500 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:19.335215 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:19.335239 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:19.337172 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:19.339143 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:19.339149 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:19.339162 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:19.339428 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:19.339693 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:19.339695 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:19.339697 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:19.339843 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:19.340272 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:19.340277 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:19.340523 (+ 246us) client_negotiation.cc:769] Sending connection context
0619 12:24:19.340683 (+ 160us) client_negotiation.cc:240] Negotiation successful
0619 12:24:19.340948 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":99,"threads_started":1}
I20260619 12:24:19.343173 7582 heartbeater.cc:347] Connected to a master server at 127.6.231.254:43389
I20260619 12:24:19.343485 7582 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:19.344038 7582 heartbeater.cc:511] Master 127.6.231.254:43389 requested a full tablet report, sending...
I20260619 12:24:19.345254 7123 ts_manager.cc:194] Registered new tserver with Master: ea9a16ca51a74393b01af8c7136171b7 (127.6.231.195:46249)
I20260619 12:24:19.345934 7123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:34987
I20260619 12:24:19.348678 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:24:19 dist-test-slave-kr9h krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871859, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:24:19 dist-test-slave-kr9h krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871859, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
I20260619 12:24:19.377010 7589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:19.366453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43520 (local address 127.6.231.193:36289)
0619 12:24:19.366736 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:19.366755 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:19.366868 (+ 113us) server_negotiation.cc:407] Connection header received
0619 12:24:19.367117 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:19.367129 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:19.367334 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:19.367495 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:19.368179 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:19.368950 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:19.369885 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:19.370353 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:19.370484 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:19.373060 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:19.373068 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:19.373082 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:19.373107 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:19.374825 (+ 1718us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:19.375328 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:19.375338 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:19.375340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:19.375416 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:19.375730 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:19.375735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:19.375736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:19.375978 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:19.376448 (+ 470us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:19.376687 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 12:24:19.376841 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":65,"threads_started":1}
W20260619 12:24:19.378118 7228 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43520
Jun 19 12:24:19 dist-test-slave-kr9h krb5kdc[7078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:24:19 dist-test-slave-kr9h krb5kdc[7078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
I20260619 12:24:19.404999 7589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:19.395501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43536 (local address 127.6.231.193:36289)
0619 12:24:19.395748 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:19.395754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:19.396058 (+ 304us) server_negotiation.cc:407] Connection header received
0619 12:24:19.396253 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:19.396259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:19.396332 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:19.396450 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:19.397323 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:19.398031 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:19.398997 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:19.399260 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:19.399358 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:19.401916 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:19.401919 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:19.401921 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:19.401946 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:19.403518 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:19.403926 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:19.403928 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:19.403929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:19.404016 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:19.404305 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:19.404307 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:19.404309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:19.404475 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:19.404575 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:19.404776 (+ 201us) server_negotiation.cc:299] Negotiation successful
0619 12:24:19.404877 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132}
I20260619 12:24:19.406267 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 7185
I20260619 12:24:19.412259 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 7319
I20260619 12:24:19.419656 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 7453
I20260619 12:24:19.424890 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 7091
2026-06-19T12:24:19Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2224 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:24:19 dist-test-slave-kr9h krb5kdc[7596](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:24:19 dist-test-slave-kr9h krb5kdc[7596](info): set up 2 sockets
Jun 19 12:24:19 dist-test-slave-kr9h krb5kdc[7596](info): commencing operation
krb5kdc: starting...
W20260619 12:24:20.529174 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.003s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:24:20 dist-test-slave-kr9h krb5kdc[7596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:24:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:24:20.653497 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:38013
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:38013
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:20.761835 7609 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:20.762164 7609 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:20.762243 7609 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:20.766291 7609 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:20.766393 7609 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:20.766433 7609 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:20.766470 7609 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:20.766530 7609 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:20.771826 7609 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:38013
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:38013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7609
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:20.772914 7609 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:20.774027 7609 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:20.782794 7615 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:20.782773 7617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:20.782786 7614 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:20.783085 7609 server_base.cc:1034] running on GCE node
I20260619 12:24:20.783780 7609 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:20.784948 7609 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:20.786119 7609 hybrid_clock.cc:648] HybridClock initialized: now 1781871860786093 us; error 37 us; skew 500 ppm
Jun 19 12:24:20 dist-test-slave-kr9h krb5kdc[7596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871860, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:20.789031 7609 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:24:20.790035 7609 webserver.cc:460] Webserver started at http://127.6.231.254:41603/ using document root <none> and password file <none>
I20260619 12:24:20.790627 7609 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:20.790725 7609 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:20.790975 7609 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:20.793125 7609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "0ffdeef873f84108846463848b13c16f"
format_stamp: "Formatted at 2026-06-19 12:24:20 on dist-test-slave-kr9h"
server_key: "100fa09aa3a128aa234fbe3d306c4820"
server_key_iv: "a4f7db3fe4752b69ec8c2410235f920f"
server_key_version: "encryptionkey@0"
I20260619 12:24:20.793751 7609 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "0ffdeef873f84108846463848b13c16f"
format_stamp: "Formatted at 2026-06-19 12:24:20 on dist-test-slave-kr9h"
server_key: "100fa09aa3a128aa234fbe3d306c4820"
server_key_iv: "a4f7db3fe4752b69ec8c2410235f920f"
server_key_version: "encryptionkey@0"
I20260619 12:24:20.797626 7609 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:24:20.800136 7624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:20.801631 7609 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:20.801764 7609 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "0ffdeef873f84108846463848b13c16f"
format_stamp: "Formatted at 2026-06-19 12:24:20 on dist-test-slave-kr9h"
server_key: "100fa09aa3a128aa234fbe3d306c4820"
server_key_iv: "a4f7db3fe4752b69ec8c2410235f920f"
server_key_version: "encryptionkey@0"
I20260619 12:24:20.801880 7609 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:20.820909 7609 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:20.824247 7609 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:20.824467 7609 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:20.831540 7609 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:38013
I20260619 12:24:20.831568 7676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:38013 every 8 connection(s)
I20260619 12:24:20.832538 7609 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:20.835489 7677 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:20.839931 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 7609
I20260619 12:24:20.840021 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:20.840277 7071 external_mini_cluster.cc:1428] Setting key 3a258ab0898b0280096594171a46620a
I20260619 12:24:20.841960 7677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f: Bootstrap starting.
I20260619 12:24:20.844892 7677 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:20.845877 7677 log.cc:824] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:20.848068 7677 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f: No bootstrap required, opened a new log
Jun 19 12:24:20 dist-test-slave-kr9h krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:20.852411 7677 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffdeef873f84108846463848b13c16f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 38013 } }
I20260619 12:24:20.852651 7677 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:20.852717 7677 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ffdeef873f84108846463848b13c16f, State: Initialized, Role: FOLLOWER
I20260619 12:24:20.853276 7677 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffdeef873f84108846463848b13c16f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 38013 } }
I20260619 12:24:20.853479 7677 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:20.853564 7677 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:20.853716 7677 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:20.854844 7677 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffdeef873f84108846463848b13c16f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 38013 } }
I20260619 12:24:20.855255 7677 leader_election.cc:302] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ffdeef873f84108846463848b13c16f; no voters:
I20260619 12:24:20.855608 7677 leader_election.cc:288] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:20.855685 7682 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:20.855877 7682 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [term 1 LEADER]: Becoming Leader. State: Replica: 0ffdeef873f84108846463848b13c16f, State: Running, Role: LEADER
I20260619 12:24:20.856192 7682 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffdeef873f84108846463848b13c16f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 38013 } }
I20260619 12:24:20.856393 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:20.841736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51674 (local address 127.6.231.254:38013)
0619 12:24:20.842160 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:20.842181 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:20.842229 (+ 48us) server_negotiation.cc:407] Connection header received
0619 12:24:20.843105 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:20.843136 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:20.843612 (+ 476us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:20.844012 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:20.844634 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:20.846346 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:20.847276 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:20.848024 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:20.848203 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:20.851081 (+ 2878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:20.851086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:20.851100 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:20.851125 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:20.852987 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:20.853539 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:20.853545 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:20.853551 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:20.853759 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:20.854118 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:20.854121 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:20.854122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:20.854488 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:20.854640 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:20.855043 (+ 403us) server_negotiation.cc:299] Negotiation successful
0619 12:24:20.855396 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":118,"threads_started":1}
I20260619 12:24:20.856981 7677 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:20.857450 7682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ffdeef873f84108846463848b13c16f. Latest consensus state: current_term: 1 leader_uuid: "0ffdeef873f84108846463848b13c16f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffdeef873f84108846463848b13c16f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 38013 } } }
I20260619 12:24:20.857589 7682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:20.857968 7683 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ffdeef873f84108846463848b13c16f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ffdeef873f84108846463848b13c16f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 38013 } } }
I20260619 12:24:20.858119 7683 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:20.858639 7686 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:20.861760 7686 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:20.868723 7686 catalog_manager.cc:1347] Generated new cluster ID: 0dcc544b7f5f45ff84152d7f4ab47c16
I20260619 12:24:20.868790 7686 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:20.891328 7686 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:20.892343 7686 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:20.905704 7686 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ffdeef873f84108846463848b13c16f: Generated new TSK 0
I20260619 12:24:20.906631 7686 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:24:20.949206 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:38013
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:21.082860 7703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:21.083133 7703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:21.083177 7703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:21.087143 7703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:21.087204 7703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:21.087265 7703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:24:21.092437 7703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:38013
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:21.093495 7703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:21.094527 7703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:21.104192 7711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:21.104274 7709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:21.104274 7708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:21.104368 7703 server_base.cc:1034] running on GCE node
I20260619 12:24:21.104812 7703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:21.105387 7703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:21.106617 7703 hybrid_clock.cc:648] HybridClock initialized: now 1781871861106594 us; error 35 us; skew 500 ppm
Jun 19 12:24:21 dist-test-slave-kr9h krb5kdc[7596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871861, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:21.109489 7703 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:24:21.110385 7703 webserver.cc:460] Webserver started at http://127.6.231.193:35451/ using document root <none> and password file <none>
I20260619 12:24:21.110948 7703 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:21.111011 7703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:21.111266 7703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:21.113308 7703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "226d602606bb484882b021f0d57580bb"
format_stamp: "Formatted at 2026-06-19 12:24:21 on dist-test-slave-kr9h"
server_key: "c89a04bb5798d23ccf3d8dc53708ac8f"
server_key_iv: "99751dc311783444794cb57db1bc3b1a"
server_key_version: "encryptionkey@0"
I20260619 12:24:21.113941 7703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "226d602606bb484882b021f0d57580bb"
format_stamp: "Formatted at 2026-06-19 12:24:21 on dist-test-slave-kr9h"
server_key: "c89a04bb5798d23ccf3d8dc53708ac8f"
server_key_iv: "99751dc311783444794cb57db1bc3b1a"
server_key_version: "encryptionkey@0"
I20260619 12:24:21.117806 7703 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:24:21.120262 7718 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:21.121604 7703 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:24:21.121735 7703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "226d602606bb484882b021f0d57580bb"
format_stamp: "Formatted at 2026-06-19 12:24:21 on dist-test-slave-kr9h"
server_key: "c89a04bb5798d23ccf3d8dc53708ac8f"
server_key_iv: "99751dc311783444794cb57db1bc3b1a"
server_key_version: "encryptionkey@0"
I20260619 12:24:21.121847 7703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:21.142107 7703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:21.145521 7703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:21.145727 7703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:21.146309 7703 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:21.147166 7703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:21.147231 7703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:21.147301 7703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:21.147344 7703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:21.156518 7703 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:46007
I20260619 12:24:21.156528 7831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:46007 every 8 connection(s)
I20260619 12:24:21.157564 7703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:24:21.166710 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 7703
I20260619 12:24:21.166826 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:24:21.167167 7071 external_mini_cluster.cc:1428] Setting key e2b02e917db2f816e517a7ef1d2286a5
Jun 19 12:24:21 dist-test-slave-kr9h krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871861, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:21.172000 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.159543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:47011 (local address 127.6.231.254:38013)
0619 12:24:21.159693 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:21.159702 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:21.160505 (+ 803us) server_negotiation.cc:407] Connection header received
0619 12:24:21.161906 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:21.161909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:21.161953 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:21.162036 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:21.163246 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.163769 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.165108 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.165345 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.165483 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.168327 (+ 2844us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:21.168330 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:21.168332 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:21.168358 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:21.169956 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:21.170494 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:21.170498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:21.170500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:21.170584 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:21.170923 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:21.170928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:21.170930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:21.171109 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:21.171213 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:21.171703 (+ 490us) server_negotiation.cc:299] Negotiation successful
0619 12:24:21.171823 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 12:24:21.172814 7834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.159891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:38013 (local address 127.6.231.193:47011)
0619 12:24:21.160337 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:21.160388 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:21.161711 (+ 1323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:21.162228 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:21.162236 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:21.162636 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:21.163113 (+ 477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.163125 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.163920 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.163923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.164984 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.164993 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.165577 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.165580 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.165728 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.166305 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:21.166322 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:21.168143 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:21.170092 (+ 1949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:21.170098 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:21.170113 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:21.170386 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:21.170678 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:21.170683 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:21.170685 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:21.170825 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:21.171211 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:21.171217 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:21.171472 (+ 255us) client_negotiation.cc:769] Sending connection context
0619 12:24:21.171665 (+ 193us) client_negotiation.cc:240] Negotiation successful
0619 12:24:21.171987 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":119,"threads_started":1}
I20260619 12:24:21.174260 7832 heartbeater.cc:347] Connected to a master server at 127.6.231.254:38013
I20260619 12:24:21.174528 7832 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:21.175134 7832 heartbeater.cc:511] Master 127.6.231.254:38013 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
I20260619 12:24:21.176882 7641 ts_manager.cc:194] Registered new tserver with Master: 226d602606bb484882b021f0d57580bb (127.6.231.193:46007)
I20260619 12:24:21.178129 7641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:47011
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:24:21.196122 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:38013
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:21.309322 7837 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:21.309616 7837 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:21.309662 7837 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:21.313692 7837 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:21.313757 7837 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:21.313818 7837 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:24:21.319428 7837 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:38013
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7837
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:21.320448 7837 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:21.321449 7837 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:21.330905 7843 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:21.330905 7845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:21.330941 7842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:21.331274 7837 server_base.cc:1034] running on GCE node
I20260619 12:24:21.331748 7837 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:21.332401 7837 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:21.333588 7837 hybrid_clock.cc:648] HybridClock initialized: now 1781871861333557 us; error 52 us; skew 500 ppm
Jun 19 12:24:21 dist-test-slave-kr9h krb5kdc[7596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871861, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:21.336404 7837 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:24:21.337214 7837 webserver.cc:460] Webserver started at http://127.6.231.194:42733/ using document root <none> and password file <none>
I20260619 12:24:21.337798 7837 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:21.337862 7837 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:21.338029 7837 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:21.340068 7837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "d9c36975ebc54c97be476c10c29efe4d"
format_stamp: "Formatted at 2026-06-19 12:24:21 on dist-test-slave-kr9h"
server_key: "d1af17957bdbd7b7cec8f5b2008271d9"
server_key_iv: "7ea61ad9692ce14c6ca3901ef4bfea6b"
server_key_version: "encryptionkey@0"
I20260619 12:24:21.340582 7837 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "d9c36975ebc54c97be476c10c29efe4d"
format_stamp: "Formatted at 2026-06-19 12:24:21 on dist-test-slave-kr9h"
server_key: "d1af17957bdbd7b7cec8f5b2008271d9"
server_key_iv: "7ea61ad9692ce14c6ca3901ef4bfea6b"
server_key_version: "encryptionkey@0"
I20260619 12:24:21.344316 7837 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 12:24:21.346944 7852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:21.348249 7837 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:21.348351 7837 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "d9c36975ebc54c97be476c10c29efe4d"
format_stamp: "Formatted at 2026-06-19 12:24:21 on dist-test-slave-kr9h"
server_key: "d1af17957bdbd7b7cec8f5b2008271d9"
server_key_iv: "7ea61ad9692ce14c6ca3901ef4bfea6b"
server_key_version: "encryptionkey@0"
I20260619 12:24:21.348445 7837 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:21.375734 7837 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:21.379165 7837 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:21.379374 7837 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:21.379968 7837 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:21.380795 7837 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:21.380841 7837 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:21.380954 7837 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:21.380972 7837 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:21.389873 7837 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:43579
I20260619 12:24:21.389894 7965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:43579 every 8 connection(s)
I20260619 12:24:21.390789 7837 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:24:21.392549 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 7837
I20260619 12:24:21.392670 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:24:21.392921 7071 external_mini_cluster.cc:1428] Setting key fb853dbf51f1fd9de4e2df982aa85bf3
Jun 19 12:24:21 dist-test-slave-kr9h krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871861, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
I20260619 12:24:21.405292 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.392604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:50069 (local address 127.6.231.254:38013)
0619 12:24:21.392749 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:21.392753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:21.393745 (+ 992us) server_negotiation.cc:407] Connection header received
0619 12:24:21.395203 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:21.395207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:21.395257 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:21.395356 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:21.396703 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.397233 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.398455 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.398686 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.398797 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.401528 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:21.401535 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:21.401538 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:21.401573 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:21.403075 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:21.403667 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:21.403671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:21.403673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:21.403756 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:21.404140 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:21.404145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:21.404147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:21.404383 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:21.404487 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:21.404993 (+ 506us) server_negotiation.cc:299] Negotiation successful
0619 12:24:21.405128 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 12:24:21.406330 7968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.393031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:38013 (local address 127.6.231.194:50069)
0619 12:24:21.393584 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:21.393641 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:21.394982 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:21.395532 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:21.395539 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:21.396067 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:21.396565 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.396578 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.397363 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.397367 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.398323 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.398332 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.398833 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.398837 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.399024 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.399636 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:21.399653 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:21.401292 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:21.403215 (+ 1923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:21.403221 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:21.403236 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:21.403560 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:21.403864 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:21.403869 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:21.403871 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:21.404042 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:21.404495 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:21.404502 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:21.404756 (+ 254us) client_negotiation.cc:769] Sending connection context
0619 12:24:21.404953 (+ 197us) client_negotiation.cc:240] Negotiation successful
0619 12:24:21.405295 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":139,"threads_started":1}
I20260619 12:24:21.407613 7966 heartbeater.cc:347] Connected to a master server at 127.6.231.254:38013
I20260619 12:24:21.407866 7966 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:21.408406 7966 heartbeater.cc:511] Master 127.6.231.254:38013 requested a full tablet report, sending...
I20260619 12:24:21.409554 7641 ts_manager.cc:194] Registered new tserver with Master: d9c36975ebc54c97be476c10c29efe4d (127.6.231.194:43579)
I20260619 12:24:21.410130 7641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:50069
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:24:21.420964 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:38013
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:40509
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:21.532799 7971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:21.533129 7971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:21.533205 7971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:21.537509 7971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:21.537592 7971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:21.537684 7971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:24:21.543212 7971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:40509
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:38013
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:21.544327 7971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:21.545398 7971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:21.555087 7979 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:21.555109 7976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:21.555106 7977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:21.555266 7971 server_base.cc:1034] running on GCE node
I20260619 12:24:21.555795 7971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:21.556481 7971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:21.557647 7971 hybrid_clock.cc:648] HybridClock initialized: now 1781871861557612 us; error 47 us; skew 500 ppm
Jun 19 12:24:21 dist-test-slave-kr9h krb5kdc[7596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871861, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:21.560509 7971 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:24:21.561386 7971 webserver.cc:460] Webserver started at http://127.6.231.195:44659/ using document root <none> and password file <none>
I20260619 12:24:21.561960 7971 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:21.562062 7971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:21.562285 7971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:21.564267 7971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "fd2c78dee8d145ea8a918c66f94e1f53"
format_stamp: "Formatted at 2026-06-19 12:24:21 on dist-test-slave-kr9h"
server_key: "43ec012d6d01e74f6b5b4520a4265231"
server_key_iv: "8e4b341a62df70f5cac867515a519983"
server_key_version: "encryptionkey@0"
I20260619 12:24:21.564814 7971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "fd2c78dee8d145ea8a918c66f94e1f53"
format_stamp: "Formatted at 2026-06-19 12:24:21 on dist-test-slave-kr9h"
server_key: "43ec012d6d01e74f6b5b4520a4265231"
server_key_iv: "8e4b341a62df70f5cac867515a519983"
server_key_version: "encryptionkey@0"
I20260619 12:24:21.568475 7971 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.003s
I20260619 12:24:21.570832 7986 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:21.571949 7971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:21.572104 7971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "fd2c78dee8d145ea8a918c66f94e1f53"
format_stamp: "Formatted at 2026-06-19 12:24:21 on dist-test-slave-kr9h"
server_key: "43ec012d6d01e74f6b5b4520a4265231"
server_key_iv: "8e4b341a62df70f5cac867515a519983"
server_key_version: "encryptionkey@0"
I20260619 12:24:21.572219 7971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:21.584388 7971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:21.587838 7971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:21.588053 7971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:21.588647 7971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:21.589507 7971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:21.589581 7971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:21.589651 7971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:21.589695 7971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:21.598412 7971 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:40717
I20260619 12:24:21.598438 8099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:40717 every 8 connection(s)
I20260619 12:24:21.599301 7971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:24:21.607407 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 7971
I20260619 12:24:21.607522 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:24:21.608109 7071 external_mini_cluster.cc:1428] Setting key 69c62b07472bcd6541716f0a8e0c781b
Jun 19 12:24:21 dist-test-slave-kr9h krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871861, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:21.612982 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.601111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:56123 (local address 127.6.231.254:38013)
0619 12:24:21.601250 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:21.601253 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:21.602134 (+ 881us) server_negotiation.cc:407] Connection header received
0619 12:24:21.603537 (+ 1403us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:21.603541 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:21.603599 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:21.603706 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:21.604879 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.605521 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.606744 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.606971 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.607081 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.609905 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:21.609911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:21.609918 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:21.609941 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:21.611277 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:21.611728 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:21.611730 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:21.611731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:21.611806 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:21.612086 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:21.612088 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:21.612089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:21.612250 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:21.612323 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:21.612769 (+ 446us) server_negotiation.cc:299] Negotiation successful
0619 12:24:21.612865 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:24:21.613860 8102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.601562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:38013 (local address 127.6.231.195:56123)
0619 12:24:21.601979 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:21.602028 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:21.603311 (+ 1283us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:21.603893 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:21.603900 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:21.604257 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:21.604732 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.604745 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.605640 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.605646 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.606606 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.606615 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.607121 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.607125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.607306 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.607919 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:21.607943 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:21.609740 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:21.611375 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:21.611379 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:21.611391 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:21.611642 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:21.611881 (+ 239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:21.611883 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:21.611885 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:21.612009 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:21.612331 (+ 322us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:21.612336 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:21.612570 (+ 234us) client_negotiation.cc:769] Sending connection context
0619 12:24:21.612724 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 12:24:21.612966 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":103,"threads_started":1}
I20260619 12:24:21.614979 8100 heartbeater.cc:347] Connected to a master server at 127.6.231.254:38013
I20260619 12:24:21.615197 8100 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:21.615629 8100 heartbeater.cc:511] Master 127.6.231.254:38013 requested a full tablet report, sending...
I20260619 12:24:21.616569 7641 ts_manager.cc:194] Registered new tserver with Master: fd2c78dee8d145ea8a918c66f94e1f53 (127.6.231.195:40717)
I20260619 12:24:21.617089 7641 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:56123
I20260619 12:24:21.621949 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:24:21.632508 7680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.625413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51682 (local address 127.6.231.254:38013)
0619 12:24:21.625615 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:21.625618 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:21.625634 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:24:21.625738 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:21.625740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:21.625783 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:21.625930 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:21.626615 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.627231 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.627996 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.628232 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.628323 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.629729 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:21.629732 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:21.629734 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:21.629758 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:21.631082 (+ 1324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:21.631469 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:21.631472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:21.631473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:21.631553 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:21.631778 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:21.631781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:21.631782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:21.631995 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:21.632100 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:21.632286 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 12:24:21.632379 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260619 12:24:21.637356 7641 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51682:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:24:21.640468 7641 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:21.655416 8116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.649556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36508 (local address 127.6.231.194:43579)
0619 12:24:21.649963 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:21.649974 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:21.650177 (+ 203us) server_negotiation.cc:407] Connection header received
0619 12:24:21.650397 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:21.650402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:21.650568 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:21.650686 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:21.651040 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.651991 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.653850 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.654421 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.654485 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.654587 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:21.655152 (+ 565us) server_negotiation.cc:299] Negotiation successful
0619 12:24:21.655247 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":98,"threads_started":1}
I20260619 12:24:21.655413 8113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.648978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35252 (local address 127.6.231.195:40717)
0619 12:24:21.649737 (+ 759us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:21.649750 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:21.649776 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:24:21.649974 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:21.649981 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:21.650181 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:21.650310 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:21.650746 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.651894 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.653829 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.654591 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.654680 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.654798 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:21.655143 (+ 345us) server_negotiation.cc:299] Negotiation successful
0619 12:24:21.655244 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":665,"thread_start_us":80,"threads_started":1}
I20260619 12:24:21.655408 8111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.648709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:40717 (local address 127.0.0.1:35252)
0619 12:24:21.649702 (+ 993us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:21.649716 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:21.649823 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:21.650501 (+ 678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:21.650503 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:21.650519 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:21.650619 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.650630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.652033 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.652036 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.653703 (+ 1667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.653713 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.654872 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.654875 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.654957 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.654990 (+ 33us) client_negotiation.cc:769] Sending connection context
0619 12:24:21.655111 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 12:24:21.655255 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":931,"mutex_wait_us":230,"thread_start_us":109,"threads_started":1}
I20260619 12:24:21.655412 8112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.648612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:46007 (local address 127.0.0.1:51720)
0619 12:24:21.649100 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:21.649146 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:21.649271 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:21.650128 (+ 857us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:21.650134 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:21.650161 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:21.650303 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.650311 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.651973 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.651977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.653570 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.653579 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.654685 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.654688 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.654927 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.655014 (+ 87us) client_negotiation.cc:769] Sending connection context
0619 12:24:21.655090 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 12:24:21.655263 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":273,"thread_start_us":98,"threads_started":1}
I20260619 12:24:21.656044 8114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.649312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51720 (local address 127.6.231.193:46007)
0619 12:24:21.649608 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:21.649626 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:21.649653 (+ 27us) server_negotiation.cc:407] Connection header received
0619 12:24:21.649731 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:21.649737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:21.649948 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:21.650450 (+ 502us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:21.650605 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.651789 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.653685 (+ 1896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.654540 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.654665 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.654801 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:21.655716 (+ 915us) server_negotiation.cc:299] Negotiation successful
0619 12:24:21.655906 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":126,"threads_started":1}
I20260619 12:24:21.656430 8115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.649368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:43579 (local address 127.0.0.1:36508)
0619 12:24:21.650097 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:21.650109 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:21.650243 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:21.650858 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:21.650860 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:21.650873 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:21.650936 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.650940 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.652117 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.652120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.653705 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.653713 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.654698 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.654700 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.654927 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.655014 (+ 87us) client_negotiation.cc:769] Sending connection context
0619 12:24:21.656211 (+ 1197us) client_negotiation.cc:240] Negotiation successful
0619 12:24:21.656318 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":676,"mutex_wait_us":194,"thread_start_us":103,"threads_started":1}
I20260619 12:24:21.658583 7766 tablet_service.cc:1467] Processing CreateTablet for tablet a3e6c3accdf64379950953cc5bd6b195 (DEFAULT_TABLE table=test-table [id=c30e40e63e524560a150ec42e39f40fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:21.658583 8034 tablet_service.cc:1467] Processing CreateTablet for tablet a3e6c3accdf64379950953cc5bd6b195 (DEFAULT_TABLE table=test-table [id=c30e40e63e524560a150ec42e39f40fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:21.659555 7766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3e6c3accdf64379950953cc5bd6b195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:21.659554 8034 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3e6c3accdf64379950953cc5bd6b195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:21.665714 8117 tablet_bootstrap.cc:492] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb: Bootstrap starting.
I20260619 12:24:21.665760 8118 tablet_bootstrap.cc:492] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53: Bootstrap starting.
I20260619 12:24:21.667974 8117 tablet_bootstrap.cc:654] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:21.668923 8118 tablet_bootstrap.cc:654] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:21.669102 8117 log.cc:824] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:21.669699 7900 tablet_service.cc:1467] Processing CreateTablet for tablet a3e6c3accdf64379950953cc5bd6b195 (DEFAULT_TABLE table=test-table [id=c30e40e63e524560a150ec42e39f40fa]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:21.670184 8118 log.cc:824] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:21.670823 7900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a3e6c3accdf64379950953cc5bd6b195. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:21.671072 8117 tablet_bootstrap.cc:492] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb: No bootstrap required, opened a new log
I20260619 12:24:21.671275 8117 ts_tablet_manager.cc:1397] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260619 12:24:21.672188 8118 tablet_bootstrap.cc:492] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53: No bootstrap required, opened a new log
I20260619 12:24:21.672442 8118 ts_tablet_manager.cc:1397] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260619 12:24:21.674598 8117 raft_consensus.cc:348] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226d602606bb484882b021f0d57580bb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46007 } } peers { permanent_uuid: "fd2c78dee8d145ea8a918c66f94e1f53" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40717 } } peers { permanent_uuid: "d9c36975ebc54c97be476c10c29efe4d" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43579 } }
I20260619 12:24:21.674825 8117 raft_consensus.cc:374] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:21.674903 8117 raft_consensus.cc:729] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 226d602606bb484882b021f0d57580bb, State: Initialized, Role: FOLLOWER
I20260619 12:24:21.675346 8117 consensus_queue.cc:260] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226d602606bb484882b021f0d57580bb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46007 } } peers { permanent_uuid: "fd2c78dee8d145ea8a918c66f94e1f53" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40717 } } peers { permanent_uuid: "d9c36975ebc54c97be476c10c29efe4d" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43579 } }
I20260619 12:24:21.676345 8117 ts_tablet_manager.cc:1428] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb: Time spent starting tablet: real 0.005s user 0.003s sys 0.002s
I20260619 12:24:21.676723 7832 heartbeater.cc:503] Master 127.6.231.254:38013 was elected leader, sending a full tablet report...
I20260619 12:24:21.676900 8118 raft_consensus.cc:348] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226d602606bb484882b021f0d57580bb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46007 } } peers { permanent_uuid: "fd2c78dee8d145ea8a918c66f94e1f53" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40717 } } peers { permanent_uuid: "d9c36975ebc54c97be476c10c29efe4d" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43579 } }
I20260619 12:24:21.677152 8118 raft_consensus.cc:374] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:21.677204 8118 raft_consensus.cc:729] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd2c78dee8d145ea8a918c66f94e1f53, State: Initialized, Role: FOLLOWER
I20260619 12:24:21.677654 8118 consensus_queue.cc:260] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226d602606bb484882b021f0d57580bb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46007 } } peers { permanent_uuid: "fd2c78dee8d145ea8a918c66f94e1f53" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40717 } } peers { permanent_uuid: "d9c36975ebc54c97be476c10c29efe4d" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43579 } }
I20260619 12:24:21.678299 8100 heartbeater.cc:503] Master 127.6.231.254:38013 was elected leader, sending a full tablet report...
I20260619 12:24:21.678607 8118 ts_tablet_manager.cc:1428] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 12:24:21.679129 8123 tablet_bootstrap.cc:492] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d: Bootstrap starting.
I20260619 12:24:21.681921 8123 tablet_bootstrap.cc:654] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:21.682641 8123 log.cc:824] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:21.684518 8123 tablet_bootstrap.cc:492] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d: No bootstrap required, opened a new log
I20260619 12:24:21.684762 8123 ts_tablet_manager.cc:1397] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260619 12:24:21.687928 8123 raft_consensus.cc:348] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226d602606bb484882b021f0d57580bb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46007 } } peers { permanent_uuid: "fd2c78dee8d145ea8a918c66f94e1f53" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40717 } } peers { permanent_uuid: "d9c36975ebc54c97be476c10c29efe4d" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43579 } }
I20260619 12:24:21.688160 8123 raft_consensus.cc:374] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:21.688244 8123 raft_consensus.cc:729] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9c36975ebc54c97be476c10c29efe4d, State: Initialized, Role: FOLLOWER
I20260619 12:24:21.688712 8123 consensus_queue.cc:260] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226d602606bb484882b021f0d57580bb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46007 } } peers { permanent_uuid: "fd2c78dee8d145ea8a918c66f94e1f53" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40717 } } peers { permanent_uuid: "d9c36975ebc54c97be476c10c29efe4d" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43579 } }
I20260619 12:24:21.689467 7966 heartbeater.cc:503] Master 127.6.231.254:38013 was elected leader, sending a full tablet report...
I20260619 12:24:21.689893 8123 ts_tablet_manager.cc:1428] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
W20260619 12:24:21.850312 8101 tablet.cc:2369] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:24:21.891883 7967 tablet.cc:2369] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:24:21.908954 7833 tablet.cc:2369] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:24:21.978235 8125 raft_consensus.cc:482] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:21.978477 8125 raft_consensus.cc:504] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226d602606bb484882b021f0d57580bb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46007 } } peers { permanent_uuid: "fd2c78dee8d145ea8a918c66f94e1f53" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40717 } } peers { permanent_uuid: "d9c36975ebc54c97be476c10c29efe4d" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43579 } }
I20260619 12:24:21.979743 8125 leader_election.cc:288] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 226d602606bb484882b021f0d57580bb (127.6.231.193:46007), fd2c78dee8d145ea8a918c66f94e1f53 (127.6.231.195:40717)
I20260619 12:24:21.983942 8113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.980091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:44909 (local address 127.6.231.195:40717)
0619 12:24:21.980389 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:21.980393 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:21.980427 (+ 34us) server_negotiation.cc:407] Connection header received
0619 12:24:21.980540 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:21.980545 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:21.980613 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:21.980696 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:21.981010 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.981662 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.982874 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.983334 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.983469 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.983527 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:21.983712 (+ 185us) server_negotiation.cc:299] Negotiation successful
0619 12:24:21.983803 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207}
I20260619 12:24:21.983955 8127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.980007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:46007 (local address 127.6.231.194:50919)
0619 12:24:21.980540 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:21.980553 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:21.980657 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:21.980929 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:21.980931 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:21.980946 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:21.981079 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.981085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.981893 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.981896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.982827 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.982840 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.983505 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.983509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.983623 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.983639 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 12:24:21.983751 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 12:24:21.983815 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"mutex_wait_us":215,"thread_start_us":85,"threads_started":1}
I20260619 12:24:21.983961 8126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.980007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:40717 (local address 127.6.231.194:44909)
0619 12:24:21.980354 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:21.980367 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:21.980454 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:21.980744 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:21.980747 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:21.980765 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:21.980899 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.980905 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.981780 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.981784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.982654 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:21.982663 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.983474 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.983478 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:21.983573 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.983603 (+ 30us) client_negotiation.cc:769] Sending connection context
0619 12:24:21.983674 (+ 71us) client_negotiation.cc:240] Negotiation successful
0619 12:24:21.983775 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":2,"thread_start_us":69,"threads_started":1}
I20260619 12:24:21.983973 8114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:21.980135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:50919 (local address 127.6.231.193:46007)
0619 12:24:21.980279 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:21.980284 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:21.980641 (+ 357us) server_negotiation.cc:407] Connection header received
0619 12:24:21.980762 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:21.980765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:21.980813 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:21.980895 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:21.981193 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.981693 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.982966 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:21.983357 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:21.983507 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:21.983562 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:21.983783 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 12:24:21.983864 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:24:21.984596 8054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3e6c3accdf64379950953cc5bd6b195" candidate_uuid: "d9c36975ebc54c97be476c10c29efe4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2c78dee8d145ea8a918c66f94e1f53" is_pre_election: true
I20260619 12:24:21.984601 7786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3e6c3accdf64379950953cc5bd6b195" candidate_uuid: "d9c36975ebc54c97be476c10c29efe4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "226d602606bb484882b021f0d57580bb" is_pre_election: true
I20260619 12:24:21.984907 8054 raft_consensus.cc:2454] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9c36975ebc54c97be476c10c29efe4d in term 0.
I20260619 12:24:21.984972 7786 raft_consensus.cc:2454] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d9c36975ebc54c97be476c10c29efe4d in term 0.
I20260619 12:24:21.985421 7856 leader_election.cc:302] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d9c36975ebc54c97be476c10c29efe4d, fd2c78dee8d145ea8a918c66f94e1f53; no voters:
I20260619 12:24:21.985785 8125 raft_consensus.cc:2781] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:21.985872 8125 raft_consensus.cc:482] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:21.985929 8125 raft_consensus.cc:3037] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:21.986950 8125 raft_consensus.cc:504] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226d602606bb484882b021f0d57580bb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46007 } } peers { permanent_uuid: "fd2c78dee8d145ea8a918c66f94e1f53" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40717 } } peers { permanent_uuid: "d9c36975ebc54c97be476c10c29efe4d" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43579 } }
I20260619 12:24:21.987358 8125 leader_election.cc:288] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [CANDIDATE]: Term 1 election: Requested vote from peers 226d602606bb484882b021f0d57580bb (127.6.231.193:46007), fd2c78dee8d145ea8a918c66f94e1f53 (127.6.231.195:40717)
I20260619 12:24:21.987730 7786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3e6c3accdf64379950953cc5bd6b195" candidate_uuid: "d9c36975ebc54c97be476c10c29efe4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "226d602606bb484882b021f0d57580bb"
I20260619 12:24:21.987776 8054 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a3e6c3accdf64379950953cc5bd6b195" candidate_uuid: "d9c36975ebc54c97be476c10c29efe4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd2c78dee8d145ea8a918c66f94e1f53"
I20260619 12:24:21.987922 8054 raft_consensus.cc:3037] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:21.987918 7786 raft_consensus.cc:3037] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:21.988978 7786 raft_consensus.cc:2454] T a3e6c3accdf64379950953cc5bd6b195 P 226d602606bb484882b021f0d57580bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9c36975ebc54c97be476c10c29efe4d in term 1.
I20260619 12:24:21.988982 8054 raft_consensus.cc:2454] T a3e6c3accdf64379950953cc5bd6b195 P fd2c78dee8d145ea8a918c66f94e1f53 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d9c36975ebc54c97be476c10c29efe4d in term 1.
I20260619 12:24:21.989285 7856 leader_election.cc:302] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 226d602606bb484882b021f0d57580bb, d9c36975ebc54c97be476c10c29efe4d, fd2c78dee8d145ea8a918c66f94e1f53; no voters:
I20260619 12:24:21.989454 8125 raft_consensus.cc:2781] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:21.989670 8125 raft_consensus.cc:686] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [term 1 LEADER]: Becoming Leader. State: Replica: d9c36975ebc54c97be476c10c29efe4d, State: Running, Role: LEADER
I20260619 12:24:21.990010 8125 consensus_queue.cc:237] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226d602606bb484882b021f0d57580bb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46007 } } peers { permanent_uuid: "fd2c78dee8d145ea8a918c66f94e1f53" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40717 } } peers { permanent_uuid: "d9c36975ebc54c97be476c10c29efe4d" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43579 } }
I20260619 12:24:21.993422 7639 catalog_manager.cc:5495] T a3e6c3accdf64379950953cc5bd6b195 P d9c36975ebc54c97be476c10c29efe4d reported cstate change: term changed from 0 to 1, leader changed from <none> to d9c36975ebc54c97be476c10c29efe4d (127.6.231.194). New cstate: current_term: 1 leader_uuid: "d9c36975ebc54c97be476c10c29efe4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "226d602606bb484882b021f0d57580bb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd2c78dee8d145ea8a918c66f94e1f53" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d9c36975ebc54c97be476c10c29efe4d" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43579 } health_report { overall_health: HEALTHY } } }
Jun 19 12:24:22 dist-test-slave-kr9h krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871860, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
I20260619 12:24:22.023654 8114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:22.014065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51734 (local address 127.6.231.193:46007)
0619 12:24:22.014270 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:22.014276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:22.014387 (+ 111us) server_negotiation.cc:407] Connection header received
0619 12:24:22.014557 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:22.014560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:22.014610 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:22.014689 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:22.015402 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:22.015982 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:22.016868 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:22.017089 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:22.017181 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:22.019859 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:22.019872 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:22.019887 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:22.019912 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:22.021813 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:22.022266 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:22.022272 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:22.022273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:22.022355 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:22.022607 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:22.022610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:22.022611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:22.022929 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:22.023102 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:22.023338 (+ 236us) server_negotiation.cc:299] Negotiation successful
0619 12:24:22.023489 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
Jun 19 12:24:22 dist-test-slave-kr9h krb5kdc[7596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871862, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:24:22 dist-test-slave-kr9h krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871862, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
I20260619 12:24:22.050210 8114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:22.041512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51738 (local address 127.6.231.193:46007)
0619 12:24:22.041652 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:22.041657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:22.041840 (+ 183us) server_negotiation.cc:407] Connection header received
0619 12:24:22.042047 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:22.042050 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:22.042093 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:22.042181 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:22.042853 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:22.043373 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:22.044234 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:22.044469 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:22.044578 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:22.047008 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:22.047012 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:22.047014 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:22.047037 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:22.048481 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:22.048894 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:22.048897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:22.048899 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:22.048983 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:22.049249 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:22.049254 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:22.049256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:22.049500 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:22.049641 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:22.049930 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 12:24:22.050048 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
Jun 19 12:24:22 dist-test-slave-kr9h krb5kdc[7596](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:24:22 dist-test-slave-kr9h krb5kdc[7596](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871862, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
I20260619 12:24:22.075899 8114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:22.067964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51748 (local address 127.6.231.193:46007)
0619 12:24:22.068101 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:22.068105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:22.068140 (+ 35us) server_negotiation.cc:407] Connection header received
0619 12:24:22.068272 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:22.068274 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:22.068334 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:22.068404 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:22.069064 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:22.069621 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:22.070444 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:22.070646 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:22.070723 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:22.072997 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:22.073000 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:22.073001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:22.073024 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:22.074499 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:22.074851 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:22.074854 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:22.074855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:22.074924 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:22.075136 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:22.075138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:22.075139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:22.075289 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:22.075405 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:22.075594 (+ 189us) server_negotiation.cc:299] Negotiation successful
0619 12:24:22.075723 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260619 12:24:22.077371 7746 tablet_service.cc:2908] Rejecting scan request for tablet a3e6c3accdf64379950953cc5bd6b195: Uninitialized: clean time has not yet been initialized
I20260619 12:24:22.079538 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 7703
I20260619 12:24:22.085074 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 7837
I20260619 12:24:22.090656 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 7971
I20260619 12:24:22.095903 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 7609
2026-06-19T12:24:22Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2672 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:24:22 dist-test-slave-kr9h krb5kdc[8142](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:24:22 dist-test-slave-kr9h krb5kdc[8142](info): set up 2 sockets
Jun 19 12:24:22 dist-test-slave-kr9h krb5kdc[8142](info): commencing operation
krb5kdc: starting...
W20260619 12:24:23.173121 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:24:23 dist-test-slave-kr9h krb5kdc[8142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:24:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:24:23.301210 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:41385
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:39781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:41385
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:23.411727 8155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:23.412067 8155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:23.412146 8155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:23.416262 8155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:23.416376 8155 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 12:24:23.416409 8155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:23.416455 8155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:23.416491 8155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:23.416529 8155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:23.421953 8155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:39781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:41385
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:41385
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:23.423122 8155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:23.424203 8155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:23.432973 8163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:23.433072 8161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:23.433022 8160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:23.433223 8155 server_base.cc:1034] running on GCE node
I20260619 12:24:23.433969 8155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:23.435153 8155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:23.436362 8155 hybrid_clock.cc:648] HybridClock initialized: now 1781871863436351 us; error 33 us; skew 500 ppm
Jun 19 12:24:23 dist-test-slave-kr9h krb5kdc[8142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:23.439366 8155 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:24:23.440336 8155 webserver.cc:460] Webserver started at http://127.6.231.254:45667/ using document root <none> and password file <none>
I20260619 12:24:23.440927 8155 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:23.441020 8155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:23.441253 8155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:23.443403 8155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "96241ec541734694a20b930cc398ef1d"
format_stamp: "Formatted at 2026-06-19 12:24:23 on dist-test-slave-kr9h"
server_key: "2b3e100032898568b485f9b9c5897020"
server_key_iv: "4fab715b26e4ba05d336f194071ad2c9"
server_key_version: "encryptionkey@0"
I20260619 12:24:23.443990 8155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "96241ec541734694a20b930cc398ef1d"
format_stamp: "Formatted at 2026-06-19 12:24:23 on dist-test-slave-kr9h"
server_key: "2b3e100032898568b485f9b9c5897020"
server_key_iv: "4fab715b26e4ba05d336f194071ad2c9"
server_key_version: "encryptionkey@0"
I20260619 12:24:23.447883 8155 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260619 12:24:23.450412 8170 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:23.451699 8155 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 12:24:23.451887 8155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "96241ec541734694a20b930cc398ef1d"
format_stamp: "Formatted at 2026-06-19 12:24:23 on dist-test-slave-kr9h"
server_key: "2b3e100032898568b485f9b9c5897020"
server_key_iv: "4fab715b26e4ba05d336f194071ad2c9"
server_key_version: "encryptionkey@0"
I20260619 12:24:23.452005 8155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:23.479182 8155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:23.482699 8155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:23.482928 8155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:23.491010 8155 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:41385
I20260619 12:24:23.491025 8232 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:41385 every 8 connection(s)
I20260619 12:24:23.491990 8155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:23.494817 8233 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:23.497721 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 8155
I20260619 12:24:23.497820 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:23.498072 7071 external_mini_cluster.cc:1428] Setting key 01143a2a18a3af429eafd393efa35a0a
I20260619 12:24:23.500937 8233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d: Bootstrap starting.
I20260619 12:24:23.503701 8233 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:23.504521 8233 log.cc:824] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:23.506631 8233 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d: No bootstrap required, opened a new log
Jun 19 12:24:23 dist-test-slave-kr9h krb5kdc[8142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871863, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:23.511165 8233 raft_consensus.cc:348] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96241ec541734694a20b930cc398ef1d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41385 } }
I20260619 12:24:23.511463 8233 raft_consensus.cc:374] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:23.511559 8233 raft_consensus.cc:729] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 96241ec541734694a20b930cc398ef1d, State: Initialized, Role: FOLLOWER
I20260619 12:24:23.512075 8233 consensus_queue.cc:260] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96241ec541734694a20b930cc398ef1d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41385 } }
I20260619 12:24:23.512240 8233 raft_consensus.cc:388] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:23.512311 8233 raft_consensus.cc:482] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:23.512418 8233 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:23.513648 8233 raft_consensus.cc:504] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96241ec541734694a20b930cc398ef1d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41385 } }
I20260619 12:24:23.514038 8233 leader_election.cc:302] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 96241ec541734694a20b930cc398ef1d; no voters:
I20260619 12:24:23.514364 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:23.499625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54686 (local address 127.6.231.254:41385)
0619 12:24:23.500076 (+ 451us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:23.500095 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:23.500139 (+ 44us) server_negotiation.cc:407] Connection header received
0619 12:24:23.500955 (+ 816us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:23.500978 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:23.501414 (+ 436us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:23.501809 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:23.502513 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:23.504018 (+ 1505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:23.504928 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:23.505532 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:23.505677 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:23.508639 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:23.508645 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:23.508666 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:23.508696 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:23.510766 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:23.511204 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:23.511210 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:23.511216 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:23.511360 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:23.511713 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:23.511716 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:23.511718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:23.512220 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:23.512432 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:23.512914 (+ 482us) server_negotiation.cc:299] Negotiation successful
0619 12:24:23.513219 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":124,"threads_started":1}
I20260619 12:24:23.514377 8233 leader_election.cc:288] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:23.514532 8238 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:23.514815 8238 raft_consensus.cc:686] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [term 1 LEADER]: Becoming Leader. State: Replica: 96241ec541734694a20b930cc398ef1d, State: Running, Role: LEADER
I20260619 12:24:23.515188 8238 consensus_queue.cc:237] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96241ec541734694a20b930cc398ef1d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41385 } }
I20260619 12:24:23.515717 8233 sys_catalog.cc:564] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:23.516611 8239 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 96241ec541734694a20b930cc398ef1d. Latest consensus state: current_term: 1 leader_uuid: "96241ec541734694a20b930cc398ef1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96241ec541734694a20b930cc398ef1d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41385 } } }
I20260619 12:24:23.516752 8239 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:23.517184 8247 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:23.517305 8240 sys_catalog.cc:455] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "96241ec541734694a20b930cc398ef1d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "96241ec541734694a20b930cc398ef1d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41385 } } }
I20260619 12:24:23.517395 8240 sys_catalog.cc:458] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:23.520519 8247 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:23.526759 8247 catalog_manager.cc:1347] Generated new cluster ID: 367bec025bfc4b01be16414fc4330941
I20260619 12:24:23.526850 8247 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:23.553468 8247 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:23.554616 8247 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:23.558902 8247 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d: Generated new TSK 0
I20260619 12:24:23.559612 8247 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:24:23.592626 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:41385
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:39781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:23.729137 8259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:23.729403 8259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:23.729519 8259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:23.733505 8259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:23.733559 8259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:23.733598 8259 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:24:23.733631 8259 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:24:23.738966 8259 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:39781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.231.254:41385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:23.739955 8259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:23.740928 8259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:23.750423 8264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:23.750447 8265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:23.750533 8259 server_base.cc:1034] running on GCE node
W20260619 12:24:23.750437 8267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:23.751201 8259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:23.751849 8259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:23.753010 8259 hybrid_clock.cc:648] HybridClock initialized: now 1781871863752978 us; error 43 us; skew 500 ppm
Jun 19 12:24:23 dist-test-slave-kr9h krb5kdc[8142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:23.759335 8259 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:24:23.760192 8259 webserver.cc:460] Webserver started at http://127.6.231.193:44187/ using document root <none> and password file <none>
I20260619 12:24:23.760720 8259 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:23.760778 8259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:23.760943 8259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:23.763012 8259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "a6908a8bc11f4f71b171b9c90de479eb"
format_stamp: "Formatted at 2026-06-19 12:24:23 on dist-test-slave-kr9h"
server_key: "9980e1d288ff292042319c96e3bae16f"
server_key_iv: "92b6d9d604994a3a88e59c79d4622fe8"
server_key_version: "encryptionkey@0"
I20260619 12:24:23.763553 8259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "a6908a8bc11f4f71b171b9c90de479eb"
format_stamp: "Formatted at 2026-06-19 12:24:23 on dist-test-slave-kr9h"
server_key: "9980e1d288ff292042319c96e3bae16f"
server_key_iv: "92b6d9d604994a3a88e59c79d4622fe8"
server_key_version: "encryptionkey@0"
I20260619 12:24:23.767294 8259 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260619 12:24:23.769845 8274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:23.771211 8259 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:23.771353 8259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "a6908a8bc11f4f71b171b9c90de479eb"
format_stamp: "Formatted at 2026-06-19 12:24:23 on dist-test-slave-kr9h"
server_key: "9980e1d288ff292042319c96e3bae16f"
server_key_iv: "92b6d9d604994a3a88e59c79d4622fe8"
server_key_version: "encryptionkey@0"
I20260619 12:24:23.771471 8259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:23.790452 8259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:23.793826 8259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:23.794049 8259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:23.795611 8259 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:23.795663 8259 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:23.795734 8259 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:23.795773 8259 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:24:23 dist-test-slave-kr9h krb5kdc[8142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:23.808018 8259 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:43263
I20260619 12:24:23.808094 8389 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:43263 every 8 connection(s)
I20260619 12:24:23.809072 8259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:24:23.810534 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:23.796689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:58811 (local address 127.6.231.254:41385)
0619 12:24:23.796826 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:23.796830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:23.798087 (+ 1257us) server_negotiation.cc:407] Connection header received
0619 12:24:23.799226 (+ 1139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:23.799230 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:23.799289 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:23.799393 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:23.800924 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:23.801645 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:23.802800 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:23.803036 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:23.803164 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:23.806455 (+ 3291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:23.806458 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:23.806460 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:23.806487 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:23.808204 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:23.808766 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:23.808770 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:23.808772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:23.808859 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:23.809249 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:23.809253 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:23.809254 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:23.809497 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:23.809603 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:23.810297 (+ 694us) server_negotiation.cc:299] Negotiation successful
0619 12:24:23.810415 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:24:23.811394 8283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:23.797309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:41385 (local address 127.6.231.193:58811)
0619 12:24:23.797932 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:23.797976 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:23.798982 (+ 1006us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:23.799568 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:23.799578 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:23.800109 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:23.800773 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:23.800793 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:23.801769 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:23.801772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:23.802658 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:23.802667 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:23.803246 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:23.803251 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:23.803519 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:23.804276 (+ 757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:23.804305 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:23.806257 (+ 1952us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:23.808339 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:23.808347 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:23.808364 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:23.808645 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:23.808960 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:23.808963 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:23.808965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:23.809108 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:23.809629 (+ 521us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:23.809640 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:23.810004 (+ 364us) client_negotiation.cc:769] Sending connection context
0619 12:24:23.810231 (+ 227us) client_negotiation.cc:240] Negotiation successful
0619 12:24:23.810506 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":243,"threads_started":1}
I20260619 12:24:23.811939 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 8259
I20260619 12:24:23.812071 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:24:23.812381 7071 external_mini_cluster.cc:1428] Setting key b3aacbf8a2d5030a681bb6bcc990cb45
I20260619 12:24:23.812808 8390 heartbeater.cc:347] Connected to a master server at 127.6.231.254:41385
I20260619 12:24:23.812927 8390 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:23.813516 8390 heartbeater.cc:511] Master 127.6.231.254:41385 requested a full tablet report, sending...
I20260619 12:24:23.815202 8187 ts_manager.cc:194] Registered new tserver with Master: a6908a8bc11f4f71b171b9c90de479eb (127.6.231.193:43263)
I20260619 12:24:23.816680 8187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:58811
I20260619 12:24:23.822908 8396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:23.815032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:41385 (local address 127.6.231.193:57739)
0619 12:24:23.815383 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:23.815397 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:23.815491 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:23.815726 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:23.815729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:23.815936 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:23.816030 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:23.816035 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:23.816918 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:23.816921 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:23.817717 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:23.817726 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:23.818174 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:23.818177 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:23.818278 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:23.818834 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:23.818853 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:23.819174 (+ 321us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:23.821238 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:23.821241 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:23.821242 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:23.821516 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:23.821843 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:23.821845 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:23.821846 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:23.821924 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:23.822417 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:23.822420 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:23.822502 (+ 82us) client_negotiation.cc:769] Sending connection context
0619 12:24:23.822599 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 12:24:23.822750 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":90,"threads_started":1}
I20260619 12:24:23.823047 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:23.815072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:57739 (local address 127.6.231.254:41385)
0619 12:24:23.815204 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:23.815208 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:23.815455 (+ 247us) server_negotiation.cc:407] Connection header received
0619 12:24:23.815567 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:23.815571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:23.815611 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:23.815712 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:23.816160 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:23.816809 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:23.817833 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:23.818065 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:23.818172 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:23.819331 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:23.819338 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:23.819344 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:23.819374 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:23.821111 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:23.821627 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:23.821634 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:23.821638 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:23.821742 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:23.822017 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:23.822022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:23.822026 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:23.822285 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:23.822401 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:23.822699 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 12:24:23.822839 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":44}
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:24:23.844199 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:41385
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:39781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:23.955355 8399 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:23.955760 8399 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:23.955843 8399 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:23.959859 8399 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:23.959950 8399 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:23.960022 8399 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:24:23.960105 8399 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:24:23.965346 8399 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:39781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.231.254:41385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8399
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:23.966459 8399 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:23.967494 8399 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:23.977461 8404 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:23.977527 8405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:23.977538 8399 server_base.cc:1034] running on GCE node
W20260619 12:24:23.977474 8407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:23.978528 8399 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:23.979314 8399 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:23.980453 8399 hybrid_clock.cc:648] HybridClock initialized: now 1781871863980434 us; error 57 us; skew 500 ppm
Jun 19 12:24:23 dist-test-slave-kr9h krb5kdc[8142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:23.983254 8399 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:24:23.984114 8399 webserver.cc:460] Webserver started at http://127.6.231.194:38775/ using document root <none> and password file <none>
I20260619 12:24:23.984622 8399 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:23.984685 8399 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:23.984848 8399 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:23.986931 8399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "177bd965fc164fa08b200984ca256841"
format_stamp: "Formatted at 2026-06-19 12:24:23 on dist-test-slave-kr9h"
server_key: "7027bde81a580001d4685faff8c13926"
server_key_iv: "d85876b714a2b11648a240c56ebc6ea3"
server_key_version: "encryptionkey@0"
I20260619 12:24:23.987461 8399 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "177bd965fc164fa08b200984ca256841"
format_stamp: "Formatted at 2026-06-19 12:24:23 on dist-test-slave-kr9h"
server_key: "7027bde81a580001d4685faff8c13926"
server_key_iv: "d85876b714a2b11648a240c56ebc6ea3"
server_key_version: "encryptionkey@0"
I20260619 12:24:23.991159 8399 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 12:24:23.993557 8414 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:23.994861 8399 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:24:23.995004 8399 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "177bd965fc164fa08b200984ca256841"
format_stamp: "Formatted at 2026-06-19 12:24:23 on dist-test-slave-kr9h"
server_key: "7027bde81a580001d4685faff8c13926"
server_key_iv: "d85876b714a2b11648a240c56ebc6ea3"
server_key_version: "encryptionkey@0"
I20260619 12:24:23.995116 8399 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:24.010046 8399 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:24.013525 8399 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:24.013762 8399 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:24.015230 8399 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:24.015286 8399 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:24.015357 8399 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:24.015390 8399 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:24:24 dist-test-slave-kr9h krb5kdc[8142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:24.026670 8399 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:34845
I20260619 12:24:24.027391 8529 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:34845 every 8 connection(s)
I20260619 12:24:24.027619 8399 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:24:24.029420 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.016250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:42031 (local address 127.6.231.254:41385)
0619 12:24:24.016413 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.016419 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.017408 (+ 989us) server_negotiation.cc:407] Connection header received
0619 12:24:24.018439 (+ 1031us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.018443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.018503 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.018564 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:24.020325 (+ 1761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.020991 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.022518 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.022766 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.022881 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.025505 (+ 2624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:24.025509 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:24.025513 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:24.025543 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:24.027174 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.027798 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.027801 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.027803 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.027893 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.028337 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.028342 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.028347 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.028575 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:24.028663 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.029167 (+ 504us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.029298 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 12:24:24.030223 8423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.016720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:41385 (local address 127.6.231.194:42031)
0619 12:24:24.017236 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:24.017286 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:24.018178 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:24.019073 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:24.019093 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:24.019559 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:24.020176 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.020191 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.021165 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.021168 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.022399 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.022408 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.022949 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.022953 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.023130 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.023798 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:24.023814 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:24.025311 (+ 1497us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:24.027299 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:24.027307 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:24.027324 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:24.027663 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:24.027985 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:24.027990 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:24.027992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:24.028188 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:24.028685 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:24.028692 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:24.028919 (+ 227us) client_negotiation.cc:769] Sending connection context
0619 12:24:24.029083 (+ 164us) client_negotiation.cc:240] Negotiation successful
0619 12:24:24.029346 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":140,"threads_started":1}
I20260619 12:24:24.030225 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 8399
I20260619 12:24:24.030361 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:24:24.030608 7071 external_mini_cluster.cc:1428] Setting key 5a0d97c230722a2bfe427585d2eb130c
I20260619 12:24:24.031699 8530 heartbeater.cc:347] Connected to a master server at 127.6.231.254:41385
I20260619 12:24:24.031853 8530 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:24.032332 8530 heartbeater.cc:511] Master 127.6.231.254:41385 requested a full tablet report, sending...
I20260619 12:24:24.033299 8186 ts_manager.cc:194] Registered new tserver with Master: 177bd965fc164fa08b200984ca256841 (127.6.231.194:34845)
I20260619 12:24:24.034163 8186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:42031
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
I20260619 12:24:24.041513 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.034057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:39175 (local address 127.6.231.254:41385)
0619 12:24:24.034501 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.034507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.034523 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:24:24.034561 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.034564 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.034608 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.034696 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:24.035192 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.035942 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.036894 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.037122 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.037238 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.038421 (+ 1183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:24.038425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:24.038427 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:24.038457 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:24.039809 (+ 1352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.040237 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.040242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.040244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.040326 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.040614 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.040619 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.040620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.040843 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:24.040955 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.041241 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.041352 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364}
I20260619 12:24:24.041527 8537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.034087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:41385 (local address 127.6.231.194:39175)
0619 12:24:24.034372 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:24.034387 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:24.034484 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:24.034731 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:24.034734 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:24.034949 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:24.035057 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.035065 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.036099 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.036102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.036763 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.036771 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.037238 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.037242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.037337 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.037969 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:24.037980 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:24.038288 (+ 308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:24.039917 (+ 1629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:24.039923 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:24.039925 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:24.040126 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:24.040423 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:24.040426 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:24.040428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:24.040514 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:24.040944 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:24.040948 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:24.041045 (+ 97us) client_negotiation.cc:769] Sending connection context
0619 12:24:24.041159 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 12:24:24.041305 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":139,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:24:24.057863 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:41385
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:39781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:24.168653 8539 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:24.168920 8539 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:24.168964 8539 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:24.172911 8539 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:24.172978 8539 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:24.173053 8539 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:24:24.173120 8539 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:24:24.178543 8539 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:39781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.231.254:41385
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8539
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:24.179631 8539 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:24.180701 8539 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:24.190441 8545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:24.190441 8547 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:24.190587 8539 server_base.cc:1034] running on GCE node
W20260619 12:24:24.190847 8544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:24.191239 8539 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:24.191869 8539 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:24.193022 8539 hybrid_clock.cc:648] HybridClock initialized: now 1781871864192990 us; error 47 us; skew 500 ppm
Jun 19 12:24:24 dist-test-slave-kr9h krb5kdc[8142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871864, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:24.195755 8539 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:24:24.196632 8539 webserver.cc:460] Webserver started at http://127.6.231.195:40873/ using document root <none> and password file <none>
I20260619 12:24:24.197193 8539 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:24.197283 8539 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:24.197564 8539 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:24.199769 8539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "809ea9dfdc2749ce81ebbe923667305b"
format_stamp: "Formatted at 2026-06-19 12:24:24 on dist-test-slave-kr9h"
server_key: "d916884830e46454e4294fe0617e1c33"
server_key_iv: "f41ad74bfd26a61890609bca9bc4e56a"
server_key_version: "encryptionkey@0"
I20260619 12:24:24.200328 8539 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "809ea9dfdc2749ce81ebbe923667305b"
format_stamp: "Formatted at 2026-06-19 12:24:24 on dist-test-slave-kr9h"
server_key: "d916884830e46454e4294fe0617e1c33"
server_key_iv: "f41ad74bfd26a61890609bca9bc4e56a"
server_key_version: "encryptionkey@0"
I20260619 12:24:24.204144 8539 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 12:24:24.206570 8554 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:24.207759 8539 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:24.207890 8539 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "809ea9dfdc2749ce81ebbe923667305b"
format_stamp: "Formatted at 2026-06-19 12:24:24 on dist-test-slave-kr9h"
server_key: "d916884830e46454e4294fe0617e1c33"
server_key_iv: "f41ad74bfd26a61890609bca9bc4e56a"
server_key_version: "encryptionkey@0"
I20260619 12:24:24.207998 8539 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:24.237561 8539 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:24.240608 8539 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:24.240823 8539 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:24.242393 8539 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:24.242466 8539 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:24.242538 8539 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:24.242588 8539 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:24:24 dist-test-slave-kr9h krb5kdc[8142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871864, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:24.253427 8539 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:45397
I20260619 12:24:24.253476 8669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:45397 every 8 connection(s)
I20260619 12:24:24.254346 8539 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:24:24.254508 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 8539
I20260619 12:24:24.254624 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:24:24.254922 7071 external_mini_cluster.cc:1428] Setting key f33ca2621ace4e7ece0365ca4b543619
I20260619 12:24:24.257190 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.243445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:37837 (local address 127.6.231.254:41385)
0619 12:24:24.243585 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.243590 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.244671 (+ 1081us) server_negotiation.cc:407] Connection header received
0619 12:24:24.245853 (+ 1182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.245857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.245916 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.246021 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:24.247406 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.248327 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.249770 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.250094 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.250226 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.252926 (+ 2700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:24.252933 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:24.252938 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:24.252970 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:24.254854 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.255478 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.255482 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.255485 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.255585 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.255999 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.256005 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.256007 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.256207 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:24.256330 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.256929 (+ 599us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.257065 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:24:24.258100 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.243995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:41385 (local address 127.6.231.195:37837)
0619 12:24:24.244508 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:24.244551 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:24.245558 (+ 1007us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:24.246190 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:24.246197 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:24.246680 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:24.247156 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.247195 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.248484 (+ 1289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.248488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.249634 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.249643 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.250261 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.250265 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.250442 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.251068 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:24.251084 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:24.252743 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:24.254994 (+ 2251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:24.255000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:24.255014 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:24.255351 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:24.255705 (+ 354us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:24.255709 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:24.255711 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:24.255918 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:24.256324 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:24.256339 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:24.256647 (+ 308us) client_negotiation.cc:769] Sending connection context
0619 12:24:24.256881 (+ 234us) client_negotiation.cc:240] Negotiation successful
0619 12:24:24.257223 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":149,"threads_started":1}
I20260619 12:24:24.259167 8670 heartbeater.cc:347] Connected to a master server at 127.6.231.254:41385
I20260619 12:24:24.259325 8670 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:24.259830 8670 heartbeater.cc:511] Master 127.6.231.254:41385 requested a full tablet report, sending...
I20260619 12:24:24.260802 8187 ts_manager.cc:194] Registered new tserver with Master: 809ea9dfdc2749ce81ebbe923667305b (127.6.231.195:45397)
I20260619 12:24:24.261523 8187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:37837
I20260619 12:24:24.268342 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.261331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:44139 (local address 127.6.231.254:41385)
0619 12:24:24.261477 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.261482 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.261685 (+ 203us) server_negotiation.cc:407] Connection header received
0619 12:24:24.261818 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.261824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.261873 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.261951 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:24.262406 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.263146 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.263920 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.264122 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.264201 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.265138 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:24.265140 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:24.265143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:24.265165 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:24.266648 (+ 1483us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.267009 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.267011 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.267013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.267080 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.267379 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.267382 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.267384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.267693 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:24.267788 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.268098 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.268210 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:24:24.268350 8676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.261275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:41385 (local address 127.6.231.195:44139)
0619 12:24:24.261611 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:24.261627 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:24.261723 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:24.261974 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:24.261979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:24.262197 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:24.262294 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.262300 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.263272 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.263274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.263821 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.263828 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.264213 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.264216 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.264284 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.264721 (+ 437us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:24.264732 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:24.265020 (+ 288us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:24.266746 (+ 1726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:24.266749 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:24.266750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:24.266931 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:24.267160 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:24.267163 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:24.267164 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:24.267258 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:24.267799 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:24.267804 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:24.267895 (+ 91us) client_negotiation.cc:769] Sending connection context
0619 12:24:24.268014 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 12:24:24.268174 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":117,"threads_started":1}
I20260619 12:24:24.269073 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:24:24 dist-test-slave-kr9h krb5kdc[8142](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871864, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:24:24 dist-test-slave-kr9h krb5kdc[8142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871864, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:24.294476 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.286645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.6.231.254:41385)
0619 12:24:24.286812 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.286816 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.286829 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:24:24.286872 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.286875 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.286914 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.286989 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:24.287701 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.288264 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.289078 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.289280 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.289354 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.291738 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:24.291741 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:24.291744 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:24.291768 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:24.293027 (+ 1259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.293400 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.293403 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.293404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.293528 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.293790 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.293792 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.293793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.293964 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:24.294066 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.294256 (+ 190us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.294352 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 12:24:24.297322 8186 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:24:24.300547 8186 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:24.315364 8687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.308823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:45397 (local address 127.0.0.1:50426)
0619 12:24:24.309284 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:24.309334 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:24.309489 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:24.310607 (+ 1118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:24.310613 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:24.310666 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:24.310970 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.310982 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.312410 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.312415 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.314000 (+ 1585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.314010 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.314814 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.314817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.314922 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.314963 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 12:24:24.315082 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 12:24:24.315206 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":481,"thread_start_us":81,"threads_started":1}
I20260619 12:24:24.315593 8690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.309577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:34845 (local address 127.0.0.1:47754)
0619 12:24:24.309906 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:24.309920 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:24.309999 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:24.310991 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:24.310994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:24.311014 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:24.311104 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.311108 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.312448 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.312452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.314064 (+ 1612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.314075 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.315152 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.315156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.315267 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.315289 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 12:24:24.315341 (+ 52us) client_negotiation.cc:240] Negotiation successful
0619 12:24:24.315411 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":73,"thread_start_us":113,"threads_started":1}
I20260619 12:24:24.315375 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.309017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50426 (local address 127.6.231.195:45397)
0619 12:24:24.310047 (+ 1030us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.310062 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.310086 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:24:24.310167 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.310173 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.310431 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.310900 (+ 469us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:24.311049 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.312267 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.314122 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.314677 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.314801 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.314971 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.315112 (+ 141us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.315248 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":925,"thread_start_us":117,"threads_started":1}
I20260619 12:24:24.316066 8689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.309081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:43263 (local address 127.0.0.1:54878)
0619 12:24:24.309698 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:24.309711 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:24.309793 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:24.310686 (+ 893us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:24.310688 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:24.310701 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:24.310914 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.310922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.312740 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.312744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.314070 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.314075 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.315152 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.315156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.315787 (+ 631us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.315815 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 12:24:24.315883 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 12:24:24.315951 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"mutex_wait_us":716,"thread_start_us":96,"threads_started":1}
I20260619 12:24:24.316152 8691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.309778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54878 (local address 127.6.231.193:43263)
0619 12:24:24.310237 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.310251 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.310270 (+ 19us) server_negotiation.cc:407] Connection header received
0619 12:24:24.310327 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.310331 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.310549 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.310712 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:24.311049 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.312609 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.314209 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.315007 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.315199 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.315343 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.315925 (+ 582us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.316018 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":87,"threads_started":1}
I20260619 12:24:24.316823 8692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.310228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47754 (local address 127.6.231.194:34845)
0619 12:24:24.310536 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.310547 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.310568 (+ 21us) server_negotiation.cc:407] Connection header received
0619 12:24:24.310638 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.310655 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.310819 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.311268 (+ 449us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:24.311376 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.312322 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.314220 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.315007 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.316384 (+ 1377us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.316558 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.316604 (+ 46us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.316687 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":105,"threads_started":1}
I20260619 12:24:24.318768 8324 tablet_service.cc:1467] Processing CreateTablet for tablet 3f106b02d58c4cceb55eb3433c2d7b67 (DEFAULT_TABLE table=test-table [id=efb0b2e5a99c4ff7a4f6d96f618924fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:24.318868 8604 tablet_service.cc:1467] Processing CreateTablet for tablet 3f106b02d58c4cceb55eb3433c2d7b67 (DEFAULT_TABLE table=test-table [id=efb0b2e5a99c4ff7a4f6d96f618924fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:24.319250 8464 tablet_service.cc:1467] Processing CreateTablet for tablet 3f106b02d58c4cceb55eb3433c2d7b67 (DEFAULT_TABLE table=test-table [id=efb0b2e5a99c4ff7a4f6d96f618924fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:24.319749 8324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f106b02d58c4cceb55eb3433c2d7b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:24.319891 8604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f106b02d58c4cceb55eb3433c2d7b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:24.320109 8464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f106b02d58c4cceb55eb3433c2d7b67. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:24.325941 8693 tablet_bootstrap.cc:492] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb: Bootstrap starting.
I20260619 12:24:24.327862 8694 tablet_bootstrap.cc:492] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841: Bootstrap starting.
I20260619 12:24:24.328399 8693 tablet_bootstrap.cc:654] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:24.329519 8693 log.cc:824] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:24.329686 8694 tablet_bootstrap.cc:654] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:24.330224 8695 tablet_bootstrap.cc:492] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b: Bootstrap starting.
I20260619 12:24:24.330485 8694 log.cc:824] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:24.331557 8693 tablet_bootstrap.cc:492] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb: No bootstrap required, opened a new log
I20260619 12:24:24.331799 8693 ts_tablet_manager.cc:1397] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 12:24:24.332284 8695 tablet_bootstrap.cc:654] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:24.332446 8694 tablet_bootstrap.cc:492] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841: No bootstrap required, opened a new log
I20260619 12:24:24.332638 8694 ts_tablet_manager.cc:1397] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 12:24:24.333105 8695 log.cc:824] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:24.335053 8695 tablet_bootstrap.cc:492] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b: No bootstrap required, opened a new log
I20260619 12:24:24.335253 8695 ts_tablet_manager.cc:1397] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.001s
I20260619 12:24:24.335240 8693 raft_consensus.cc:348] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } }
I20260619 12:24:24.335518 8693 raft_consensus.cc:374] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:24.335573 8693 raft_consensus.cc:729] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6908a8bc11f4f71b171b9c90de479eb, State: Initialized, Role: FOLLOWER
I20260619 12:24:24.336010 8693 consensus_queue.cc:260] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } }
I20260619 12:24:24.336356 8694 raft_consensus.cc:348] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } }
I20260619 12:24:24.336585 8390 heartbeater.cc:503] Master 127.6.231.254:41385 was elected leader, sending a full tablet report...
I20260619 12:24:24.336594 8694 raft_consensus.cc:374] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:24.336637 8694 raft_consensus.cc:729] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177bd965fc164fa08b200984ca256841, State: Initialized, Role: FOLLOWER
I20260619 12:24:24.336958 8693 ts_tablet_manager.cc:1428] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 12:24:24.337620 8694 consensus_queue.cc:260] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } }
I20260619 12:24:24.338613 8694 ts_tablet_manager.cc:1428] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 12:24:24.338945 8530 heartbeater.cc:503] Master 127.6.231.254:41385 was elected leader, sending a full tablet report...
I20260619 12:24:24.338994 8695 raft_consensus.cc:348] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } }
I20260619 12:24:24.339259 8695 raft_consensus.cc:374] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:24.339318 8695 raft_consensus.cc:729] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 809ea9dfdc2749ce81ebbe923667305b, State: Initialized, Role: FOLLOWER
I20260619 12:24:24.339701 8695 consensus_queue.cc:260] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } }
I20260619 12:24:24.340341 8670 heartbeater.cc:503] Master 127.6.231.254:41385 was elected leader, sending a full tablet report...
I20260619 12:24:24.341567 8695 ts_tablet_manager.cc:1428] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260619 12:24:24.467281 8701 raft_consensus.cc:482] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:24.467474 8701 raft_consensus.cc:504] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } }
I20260619 12:24:24.468593 8701 leader_election.cc:288] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6908a8bc11f4f71b171b9c90de479eb (127.6.231.193:43263), 177bd965fc164fa08b200984ca256841 (127.6.231.194:34845)
I20260619 12:24:24.472754 8691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.469075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:56111 (local address 127.6.231.193:43263)
0619 12:24:24.469225 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.469231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.469263 (+ 32us) server_negotiation.cc:407] Connection header received
0619 12:24:24.469323 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.469325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.469362 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.469465 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:24.469764 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.470431 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.471548 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.472177 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.472315 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.472368 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.472567 (+ 199us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.472621 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 12:24:24.472756 8563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.468879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:43263 (local address 127.6.231.195:56111)
0619 12:24:24.469027 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:24.469045 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:24.469189 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:24.469481 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:24.469484 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:24.469501 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:24.469619 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.469626 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.470544 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.470547 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.471423 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.471432 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.472315 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.472318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.472442 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.472467 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 12:24:24.472548 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 12:24:24.472616 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 12:24:24.473500 8692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.468996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:35815 (local address 127.6.231.194:34845)
0619 12:24:24.469127 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.469134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.469619 (+ 485us) server_negotiation.cc:407] Connection header received
0619 12:24:24.469705 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.469708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.469759 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.469821 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:24.470097 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.470672 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.472304 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.472902 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.473022 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.473068 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.473285 (+ 217us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.473359 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:24:24.473512 8702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.468879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:34845 (local address 127.6.231.195:35815)
0619 12:24:24.469521 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:24.469533 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:24.469625 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:24.469850 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:24.469853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:24.469870 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:24.469989 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.469996 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.470784 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.470787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.472188 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.472199 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.473045 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.473049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.473145 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.473174 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 12:24:24.473268 (+ 94us) client_negotiation.cc:240] Negotiation successful
0619 12:24:24.473334 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":247,"thread_start_us":140,"threads_started":1}
I20260619 12:24:24.473510 8344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f106b02d58c4cceb55eb3433c2d7b67" candidate_uuid: "809ea9dfdc2749ce81ebbe923667305b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6908a8bc11f4f71b171b9c90de479eb" is_pre_election: true
I20260619 12:24:24.473889 8344 raft_consensus.cc:2454] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 809ea9dfdc2749ce81ebbe923667305b in term 0.
I20260619 12:24:24.474080 8484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f106b02d58c4cceb55eb3433c2d7b67" candidate_uuid: "809ea9dfdc2749ce81ebbe923667305b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177bd965fc164fa08b200984ca256841" is_pre_election: true
I20260619 12:24:24.474380 8484 raft_consensus.cc:2454] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 809ea9dfdc2749ce81ebbe923667305b in term 0.
I20260619 12:24:24.474387 8555 leader_election.cc:302] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 809ea9dfdc2749ce81ebbe923667305b, a6908a8bc11f4f71b171b9c90de479eb; no voters:
I20260619 12:24:24.474659 8701 raft_consensus.cc:2781] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:24.474749 8701 raft_consensus.cc:482] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:24.474821 8701 raft_consensus.cc:3037] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:24.475945 8701 raft_consensus.cc:504] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } }
I20260619 12:24:24.476343 8701 leader_election.cc:288] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [CANDIDATE]: Term 1 election: Requested vote from peers a6908a8bc11f4f71b171b9c90de479eb (127.6.231.193:43263), 177bd965fc164fa08b200984ca256841 (127.6.231.194:34845)
I20260619 12:24:24.476704 8484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f106b02d58c4cceb55eb3433c2d7b67" candidate_uuid: "809ea9dfdc2749ce81ebbe923667305b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177bd965fc164fa08b200984ca256841"
I20260619 12:24:24.476734 8344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f106b02d58c4cceb55eb3433c2d7b67" candidate_uuid: "809ea9dfdc2749ce81ebbe923667305b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6908a8bc11f4f71b171b9c90de479eb"
I20260619 12:24:24.476830 8484 raft_consensus.cc:3037] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:24.476845 8344 raft_consensus.cc:3037] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:24.478165 8484 raft_consensus.cc:2454] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 809ea9dfdc2749ce81ebbe923667305b in term 1.
I20260619 12:24:24.478157 8344 raft_consensus.cc:2454] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 809ea9dfdc2749ce81ebbe923667305b in term 1.
I20260619 12:24:24.478467 8555 leader_election.cc:302] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 809ea9dfdc2749ce81ebbe923667305b, a6908a8bc11f4f71b171b9c90de479eb; no voters:
I20260619 12:24:24.478633 8701 raft_consensus.cc:2781] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:24.478856 8701 raft_consensus.cc:686] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [term 1 LEADER]: Becoming Leader. State: Replica: 809ea9dfdc2749ce81ebbe923667305b, State: Running, Role: LEADER
I20260619 12:24:24.479243 8701 consensus_queue.cc:237] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } }
I20260619 12:24:24.482343 8186 catalog_manager.cc:5495] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b reported cstate change: term changed from 0 to 1, leader changed from <none> to 809ea9dfdc2749ce81ebbe923667305b (127.6.231.195). New cstate: current_term: 1 leader_uuid: "809ea9dfdc2749ce81ebbe923667305b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } health_report { overall_health: UNKNOWN } } }
W20260619 12:24:24.505558 8671 tablet.cc:2369] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:24:24.508589 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.504510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50442 (local address 127.6.231.195:45397)
0619 12:24:24.504743 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.504749 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.504824 (+ 75us) server_negotiation.cc:407] Connection header received
0619 12:24:24.505068 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.505071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.505125 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.505257 (+ 132us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:24.505544 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.506125 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.507026 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.507258 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.507458 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.507778 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:24.508179 (+ 401us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:24.508315 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.508387 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.508449 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155}
I20260619 12:24:24.514868 8484 raft_consensus.cc:1261] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841 [term 1 FOLLOWER]: Refusing update from remote peer 809ea9dfdc2749ce81ebbe923667305b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:24.514868 8344 raft_consensus.cc:1261] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb [term 1 FOLLOWER]: Refusing update from remote peer 809ea9dfdc2749ce81ebbe923667305b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:24.515563 8703 consensus_queue.cc:1035] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:24.515751 8701 consensus_queue.cc:1035] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:24.526306 8708 mvcc.cc:205] Tried to move back new op lower bound from 7298547157045145600 to 7298547156909633536. Current Snapshot: MvccSnapshot[applied={T|T < 7298547157045145600}]
I20260619 12:24:24.536103 8187 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:24.536269 8187 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:24.538807 8187 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d: Sending DeleteTablet for 3 replicas of tablet 3f106b02d58c4cceb55eb3433c2d7b67
I20260619 12:24:24.539628 8604 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f106b02d58c4cceb55eb3433c2d7b67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:24 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:50426
I20260619 12:24:24.539633 8464 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f106b02d58c4cceb55eb3433c2d7b67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:24 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:47754
I20260619 12:24:24.539742 8324 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f106b02d58c4cceb55eb3433c2d7b67 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:24 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:54878
I20260619 12:24:24.540083 8716 tablet_replica.cc:330] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841: stopping tablet replica
I20260619 12:24:24.540083 8717 tablet_replica.cc:330] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b: stopping tablet replica
I20260619 12:24:24.540385 8717 raft_consensus.cc:2229] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:24:24.540385 8716 raft_consensus.cc:2229] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:24.540685 8716 raft_consensus.cc:2258] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:24.540760 8717 raft_consensus.cc:2258] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:24.541201 8718 tablet_replica.cc:330] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb: stopping tablet replica
I20260619 12:24:24.541527 8718 raft_consensus.cc:2229] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:24.541739 8187 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 12:24:24.541949 8718 raft_consensus.cc:2258] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 12:24:24.542165 8187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:24.542429 8716 ts_tablet_manager.cc:1905] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:24.543174 8718 ts_tablet_manager.cc:1905] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:24.545193 8716 ts_tablet_manager.cc:1918] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:24.545285 8716 log.cc:1196] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/3f106b02d58c4cceb55eb3433c2d7b67
I20260619 12:24:24.545615 8716 ts_tablet_manager.cc:1939] T 3f106b02d58c4cceb55eb3433c2d7b67 P 177bd965fc164fa08b200984ca256841: Deleting consensus metadata
I20260619 12:24:24.545941 8718 ts_tablet_manager.cc:1918] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:24.546039 8718 log.cc:1196] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/wal/wals/3f106b02d58c4cceb55eb3433c2d7b67
I20260619 12:24:24.546324 8718 ts_tablet_manager.cc:1939] T 3f106b02d58c4cceb55eb3433c2d7b67 P a6908a8bc11f4f71b171b9c90de479eb: Deleting consensus metadata
I20260619 12:24:24.546904 8173 catalog_manager.cc:4841] TS 177bd965fc164fa08b200984ca256841 (127.6.231.194:34845): tablet 3f106b02d58c4cceb55eb3433c2d7b67 (table test-table [id=efb0b2e5a99c4ff7a4f6d96f618924fc]) successfully deleted
I20260619 12:24:24.547883 8171 catalog_manager.cc:4841] TS a6908a8bc11f4f71b171b9c90de479eb (127.6.231.193:43263): tablet 3f106b02d58c4cceb55eb3433c2d7b67 (table test-table [id=efb0b2e5a99c4ff7a4f6d96f618924fc]) successfully deleted
I20260619 12:24:24.548486 8717 ts_tablet_manager.cc:1905] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:24.548733 8464 tablet_service.cc:1467] Processing CreateTablet for tablet 97da2de37080440da8358f0b6b46d6fa (DEFAULT_TABLE table=test-table [id=dc979f3bea0c4aab94461afe12619df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:24.548820 8604 tablet_service.cc:1467] Processing CreateTablet for tablet 97da2de37080440da8358f0b6b46d6fa (DEFAULT_TABLE table=test-table [id=dc979f3bea0c4aab94461afe12619df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:24.549029 8464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97da2de37080440da8358f0b6b46d6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:24.549115 8604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97da2de37080440da8358f0b6b46d6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:24.551139 8717 ts_tablet_manager.cc:1918] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:24.551240 8717 log.cc:1196] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/3f106b02d58c4cceb55eb3433c2d7b67
I20260619 12:24:24.551276 8694 tablet_bootstrap.cc:492] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841: Bootstrap starting.
I20260619 12:24:24.551426 8695 tablet_bootstrap.cc:492] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b: Bootstrap starting.
I20260619 12:24:24.551553 8717 ts_tablet_manager.cc:1939] T 3f106b02d58c4cceb55eb3433c2d7b67 P 809ea9dfdc2749ce81ebbe923667305b: Deleting consensus metadata
I20260619 12:24:24.552225 8694 tablet_bootstrap.cc:654] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:24.552361 8695 tablet_bootstrap.cc:654] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:24.553153 8173 catalog_manager.cc:4841] TS 809ea9dfdc2749ce81ebbe923667305b (127.6.231.195:45397): tablet 3f106b02d58c4cceb55eb3433c2d7b67 (table test-table [id=efb0b2e5a99c4ff7a4f6d96f618924fc]) successfully deleted
I20260619 12:24:24.553151 8324 tablet_service.cc:1467] Processing CreateTablet for tablet 97da2de37080440da8358f0b6b46d6fa (DEFAULT_TABLE table=test-table [id=dc979f3bea0c4aab94461afe12619df7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:24.553424 8324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 97da2de37080440da8358f0b6b46d6fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:24.553741 8694 tablet_bootstrap.cc:492] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841: No bootstrap required, opened a new log
I20260619 12:24:24.553825 8694 ts_tablet_manager.cc:1397] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:24:24.554236 8694 raft_consensus.cc:348] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.554348 8694 raft_consensus.cc:374] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:24.554378 8694 raft_consensus.cc:729] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177bd965fc164fa08b200984ca256841, State: Initialized, Role: FOLLOWER
I20260619 12:24:24.554500 8694 consensus_queue.cc:260] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.554780 8694 ts_tablet_manager.cc:1428] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:24.555167 8695 tablet_bootstrap.cc:492] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b: No bootstrap required, opened a new log
I20260619 12:24:24.555258 8695 ts_tablet_manager.cc:1397] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 12:24:24.555714 8695 raft_consensus.cc:348] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.555774 8693 tablet_bootstrap.cc:492] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb: Bootstrap starting.
I20260619 12:24:24.555811 8695 raft_consensus.cc:374] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:24.555841 8695 raft_consensus.cc:729] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 809ea9dfdc2749ce81ebbe923667305b, State: Initialized, Role: FOLLOWER
I20260619 12:24:24.555953 8695 consensus_queue.cc:260] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.556182 8695 ts_tablet_manager.cc:1428] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:24.556768 8693 tablet_bootstrap.cc:654] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:24.558254 8693 tablet_bootstrap.cc:492] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb: No bootstrap required, opened a new log
I20260619 12:24:24.558359 8693 ts_tablet_manager.cc:1397] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:24:24.559111 8693 raft_consensus.cc:348] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.559237 8693 raft_consensus.cc:374] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:24.559273 8693 raft_consensus.cc:729] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6908a8bc11f4f71b171b9c90de479eb, State: Initialized, Role: FOLLOWER
I20260619 12:24:24.559445 8693 consensus_queue.cc:260] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.559804 8693 ts_tablet_manager.cc:1428] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
W20260619 12:24:24.560786 8391 tablet.cc:2369] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:24:24.667877 8712 raft_consensus.cc:482] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:24.668044 8712 raft_consensus.cc:504] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.668514 8712 leader_election.cc:288] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6908a8bc11f4f71b171b9c90de479eb (127.6.231.193:43263), 177bd965fc164fa08b200984ca256841 (127.6.231.194:34845)
I20260619 12:24:24.669059 8344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97da2de37080440da8358f0b6b46d6fa" candidate_uuid: "809ea9dfdc2749ce81ebbe923667305b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6908a8bc11f4f71b171b9c90de479eb" is_pre_election: true
I20260619 12:24:24.669056 8484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97da2de37080440da8358f0b6b46d6fa" candidate_uuid: "809ea9dfdc2749ce81ebbe923667305b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177bd965fc164fa08b200984ca256841" is_pre_election: true
I20260619 12:24:24.669253 8484 raft_consensus.cc:2454] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 809ea9dfdc2749ce81ebbe923667305b in term 0.
I20260619 12:24:24.669265 8344 raft_consensus.cc:2454] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 809ea9dfdc2749ce81ebbe923667305b in term 0.
I20260619 12:24:24.669624 8557 leader_election.cc:302] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177bd965fc164fa08b200984ca256841, 809ea9dfdc2749ce81ebbe923667305b; no voters:
I20260619 12:24:24.669817 8712 raft_consensus.cc:2781] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:24.669881 8712 raft_consensus.cc:482] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:24.669946 8712 raft_consensus.cc:3037] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:24.671020 8712 raft_consensus.cc:504] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.671432 8712 leader_election.cc:288] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [CANDIDATE]: Term 1 election: Requested vote from peers a6908a8bc11f4f71b171b9c90de479eb (127.6.231.193:43263), 177bd965fc164fa08b200984ca256841 (127.6.231.194:34845)
I20260619 12:24:24.671842 8344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97da2de37080440da8358f0b6b46d6fa" candidate_uuid: "809ea9dfdc2749ce81ebbe923667305b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6908a8bc11f4f71b171b9c90de479eb"
I20260619 12:24:24.671852 8484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "97da2de37080440da8358f0b6b46d6fa" candidate_uuid: "809ea9dfdc2749ce81ebbe923667305b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177bd965fc164fa08b200984ca256841"
I20260619 12:24:24.671978 8484 raft_consensus.cc:3037] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:24.671985 8344 raft_consensus.cc:3037] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:24.672916 8344 raft_consensus.cc:2454] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 809ea9dfdc2749ce81ebbe923667305b in term 1.
I20260619 12:24:24.673152 8484 raft_consensus.cc:2454] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 809ea9dfdc2749ce81ebbe923667305b in term 1.
I20260619 12:24:24.673264 8555 leader_election.cc:302] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 809ea9dfdc2749ce81ebbe923667305b, a6908a8bc11f4f71b171b9c90de479eb; no voters:
I20260619 12:24:24.673523 8712 raft_consensus.cc:2781] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:24.673635 8712 raft_consensus.cc:686] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [term 1 LEADER]: Becoming Leader. State: Replica: 809ea9dfdc2749ce81ebbe923667305b, State: Running, Role: LEADER
I20260619 12:24:24.673784 8712 consensus_queue.cc:237] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.675559 8187 catalog_manager.cc:5495] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b reported cstate change: term changed from 0 to 1, leader changed from <none> to 809ea9dfdc2749ce81ebbe923667305b (127.6.231.195). New cstate: current_term: 1 leader_uuid: "809ea9dfdc2749ce81ebbe923667305b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } health_report { overall_health: HEALTHY } } }
Jun 19 12:24:24 dist-test-slave-kr9h krb5kdc[8142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871863, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:24.708559 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.699607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54696 (local address 127.6.231.254:41385)
0619 12:24:24.699760 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.699765 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.699938 (+ 173us) server_negotiation.cc:407] Connection header received
0619 12:24:24.700139 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.700141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.700187 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.700279 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:24.700954 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.701582 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.702399 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.702616 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.702726 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.705130 (+ 2404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:24.705135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:24.705138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:24.705166 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:24.706801 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.707294 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.707297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.707298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.707374 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.707706 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.707709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.707710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.707908 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:24.707996 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.708289 (+ 293us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.708446 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":57}
I20260619 12:24:24.708561 8728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.699534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:41385 (local address 127.0.0.1:54696)
0619 12:24:24.699847 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:24.699859 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:24.699995 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:24.700298 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:24.700301 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:24.700731 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:24.700833 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.700839 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.701689 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.701695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.702270 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.702278 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.702729 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.702732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.702802 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.703436 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:24.703449 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:24.704974 (+ 1525us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:24.706920 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:24.706929 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:24.706934 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:24.707192 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:24.707483 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:24.707486 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:24.707487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:24.707577 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:24.708010 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:24.708016 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:24.708133 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 12:24:24.708228 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 12:24:24.708369 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":119,"threads_started":1}
I20260619 12:24:24.712850 8187 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:54696:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 12:24:24.713755 8187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:24.719173 8464 tablet_service.cc:1467] Processing CreateTablet for tablet e5f4616899f245e68318cf36397c9348 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=50626afa607b4428acae62748aee1ebd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:24:24.719156 8324 tablet_service.cc:1467] Processing CreateTablet for tablet e5f4616899f245e68318cf36397c9348 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=50626afa607b4428acae62748aee1ebd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:24:24.719588 8464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f4616899f245e68318cf36397c9348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:24.719621 8324 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f4616899f245e68318cf36397c9348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:24.722386 8694 tablet_bootstrap.cc:492] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841: Bootstrap starting.
I20260619 12:24:24.722815 8604 tablet_service.cc:1467] Processing CreateTablet for tablet e5f4616899f245e68318cf36397c9348 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=50626afa607b4428acae62748aee1ebd]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:24:24.723121 8604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e5f4616899f245e68318cf36397c9348. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:24.723697 8694 tablet_bootstrap.cc:654] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:24.724318 8693 tablet_bootstrap.cc:492] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb: Bootstrap starting.
I20260619 12:24:24.725260 8693 tablet_bootstrap.cc:654] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:24.725972 8694 tablet_bootstrap.cc:492] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841: No bootstrap required, opened a new log
I20260619 12:24:24.726085 8694 ts_tablet_manager.cc:1397] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 12:24:24.726174 8695 tablet_bootstrap.cc:492] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b: Bootstrap starting.
I20260619 12:24:24.726446 8693 tablet_bootstrap.cc:492] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb: No bootstrap required, opened a new log
I20260619 12:24:24.726506 8693 ts_tablet_manager.cc:1397] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260619 12:24:24.726518 8694 raft_consensus.cc:348] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.726643 8694 raft_consensus.cc:374] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:24.726675 8694 raft_consensus.cc:729] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 177bd965fc164fa08b200984ca256841, State: Initialized, Role: FOLLOWER
I20260619 12:24:24.726801 8694 consensus_queue.cc:260] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.726857 8693 raft_consensus.cc:348] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.726984 8693 raft_consensus.cc:374] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:24.727016 8693 raft_consensus.cc:729] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6908a8bc11f4f71b171b9c90de479eb, State: Initialized, Role: FOLLOWER
I20260619 12:24:24.727069 8695 tablet_bootstrap.cc:654] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:24.727106 8694 ts_tablet_manager.cc:1428] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:24:24.727067 8700 tablet_replica.cc:439] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } }
I20260619 12:24:24.727129 8693 consensus_queue.cc:260] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.727218 8700 tablet_replica.cc:442] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:24.727399 8693 ts_tablet_manager.cc:1428] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 12:24:24.727396 8699 tablet_replica.cc:439] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } }
I20260619 12:24:24.728264 8699 tablet_replica.cc:442] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:24.728268 8695 tablet_bootstrap.cc:492] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b: No bootstrap required, opened a new log
I20260619 12:24:24.728340 8695 ts_tablet_manager.cc:1397] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:24.728786 8695 raft_consensus.cc:348] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.728901 8695 raft_consensus.cc:374] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:24.728933 8695 raft_consensus.cc:729] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 809ea9dfdc2749ce81ebbe923667305b, State: Initialized, Role: FOLLOWER
I20260619 12:24:24.729060 8695 consensus_queue.cc:260] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.729333 8695 ts_tablet_manager.cc:1428] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:24:24.729301 8712 tablet_replica.cc:439] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } }
I20260619 12:24:24.729467 8712 tablet_replica.cc:442] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b: This TxnStatusTablet replica's current role is: FOLLOWER
W20260619 12:24:24.779558 8531 tablet.cc:2369] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:24:24.820715 8712 raft_consensus.cc:482] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:24.820906 8712 raft_consensus.cc:504] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.821382 8712 leader_election.cc:288] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a6908a8bc11f4f71b171b9c90de479eb (127.6.231.193:43263), 177bd965fc164fa08b200984ca256841 (127.6.231.194:34845)
I20260619 12:24:24.821831 8344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f4616899f245e68318cf36397c9348" candidate_uuid: "809ea9dfdc2749ce81ebbe923667305b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6908a8bc11f4f71b171b9c90de479eb" is_pre_election: true
I20260619 12:24:24.821853 8484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f4616899f245e68318cf36397c9348" candidate_uuid: "809ea9dfdc2749ce81ebbe923667305b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177bd965fc164fa08b200984ca256841" is_pre_election: true
I20260619 12:24:24.821973 8344 raft_consensus.cc:2454] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 809ea9dfdc2749ce81ebbe923667305b in term 0.
I20260619 12:24:24.822012 8484 raft_consensus.cc:2454] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 809ea9dfdc2749ce81ebbe923667305b in term 0.
I20260619 12:24:24.822261 8555 leader_election.cc:302] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 809ea9dfdc2749ce81ebbe923667305b, a6908a8bc11f4f71b171b9c90de479eb; no voters:
I20260619 12:24:24.822420 8712 raft_consensus.cc:2781] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:24.822485 8712 raft_consensus.cc:482] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:24.822549 8712 raft_consensus.cc:3037] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:24.823571 8712 raft_consensus.cc:504] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.823956 8712 leader_election.cc:288] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [CANDIDATE]: Term 1 election: Requested vote from peers a6908a8bc11f4f71b171b9c90de479eb (127.6.231.193:43263), 177bd965fc164fa08b200984ca256841 (127.6.231.194:34845)
I20260619 12:24:24.824280 8344 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f4616899f245e68318cf36397c9348" candidate_uuid: "809ea9dfdc2749ce81ebbe923667305b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a6908a8bc11f4f71b171b9c90de479eb"
I20260619 12:24:24.824343 8484 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e5f4616899f245e68318cf36397c9348" candidate_uuid: "809ea9dfdc2749ce81ebbe923667305b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "177bd965fc164fa08b200984ca256841"
I20260619 12:24:24.824411 8344 raft_consensus.cc:3037] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:24.824487 8484 raft_consensus.cc:3037] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:24.825335 8484 raft_consensus.cc:2454] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 809ea9dfdc2749ce81ebbe923667305b in term 1.
I20260619 12:24:24.825399 8344 raft_consensus.cc:2454] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 809ea9dfdc2749ce81ebbe923667305b in term 1.
I20260619 12:24:24.825736 8557 leader_election.cc:302] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 177bd965fc164fa08b200984ca256841, 809ea9dfdc2749ce81ebbe923667305b; no voters:
I20260619 12:24:24.825938 8712 raft_consensus.cc:2781] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:24.826040 8712 raft_consensus.cc:686] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [term 1 LEADER]: Becoming Leader. State: Replica: 809ea9dfdc2749ce81ebbe923667305b, State: Running, Role: LEADER
I20260619 12:24:24.826186 8712 consensus_queue.cc:237] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } }
I20260619 12:24:24.826870 8701 tablet_replica.cc:439] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b: TxnStatusTablet state changed. Reason: New leader 809ea9dfdc2749ce81ebbe923667305b. Latest consensus state: current_term: 1 leader_uuid: "809ea9dfdc2749ce81ebbe923667305b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } }
I20260619 12:24:24.826968 8701 tablet_replica.cc:442] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:24:24.827215 8733 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 12:24:24.827382 8733 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 12:24:24.828014 8186 catalog_manager.cc:5495] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b reported cstate change: term changed from 0 to 1, leader changed from <none> to 809ea9dfdc2749ce81ebbe923667305b (127.6.231.195). New cstate: current_term: 1 leader_uuid: "809ea9dfdc2749ce81ebbe923667305b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } health_report { overall_health: HEALTHY } } }
I20260619 12:24:24.850359 8728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.846925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:45397 (local address 127.0.0.1:50456)
0619 12:24:24.847087 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:24.847128 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:24.847248 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:24.847558 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:24.847561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:24.847568 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:24:24.847683 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.847688 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.848413 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.848418 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.849033 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.849041 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.849560 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.849564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.849663 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.849710 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:24:24.850034 (+ 324us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:24:24.850051 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 12:24:24.850146 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 12:24:24.850238 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 12:24:24.850359 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.847072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50456 (local address 127.6.231.195:45397)
0619 12:24:24.847213 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.847217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.847232 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:24:24.847393 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.847396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.847450 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.847536 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:24.847795 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.848294 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.849168 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.849387 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.849548 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.849859 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:24.849945 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:24.850028 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.850158 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.850212 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 12:24:24.852946 8344 raft_consensus.cc:1261] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb [term 1 FOLLOWER]: Refusing update from remote peer 809ea9dfdc2749ce81ebbe923667305b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:24.852995 8484 raft_consensus.cc:1261] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841 [term 1 FOLLOWER]: Refusing update from remote peer 809ea9dfdc2749ce81ebbe923667305b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:24.853471 8701 consensus_queue.cc:1035] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:24.853714 8712 consensus_queue.cc:1035] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:24.855041 8700 tablet_replica.cc:439] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841: TxnStatusTablet state changed. Reason: New leader 809ea9dfdc2749ce81ebbe923667305b. Latest consensus state: current_term: 1 leader_uuid: "809ea9dfdc2749ce81ebbe923667305b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } }
I20260619 12:24:24.855171 8700 tablet_replica.cc:442] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:24.855118 8699 tablet_replica.cc:439] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb: TxnStatusTablet state changed. Reason: New leader 809ea9dfdc2749ce81ebbe923667305b. Latest consensus state: current_term: 1 leader_uuid: "809ea9dfdc2749ce81ebbe923667305b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } }
I20260619 12:24:24.855263 8699 tablet_replica.cc:442] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:24.856778 8712 tablet_replica.cc:439] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "809ea9dfdc2749ce81ebbe923667305b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } }
I20260619 12:24:24.856889 8712 tablet_replica.cc:442] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:24:24.858891 8699 tablet_replica.cc:439] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "809ea9dfdc2749ce81ebbe923667305b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } }
I20260619 12:24:24.859007 8699 tablet_replica.cc:442] T e5f4616899f245e68318cf36397c9348 P a6908a8bc11f4f71b171b9c90de479eb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:24.859467 8703 tablet_replica.cc:439] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "809ea9dfdc2749ce81ebbe923667305b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } }
I20260619 12:24:24.859612 8703 tablet_replica.cc:442] T e5f4616899f245e68318cf36397c9348 P 809ea9dfdc2749ce81ebbe923667305b: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:24:24.860908 8707 mvcc.cc:205] Tried to move back new op lower bound from 7298547158433480704 to 7298547158329618432. Current Snapshot: MvccSnapshot[applied={T|T < 7298547158433480704}]
I20260619 12:24:24.861088 8700 tablet_replica.cc:439] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "809ea9dfdc2749ce81ebbe923667305b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 } } peers { permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 } } peers { permanent_uuid: "809ea9dfdc2749ce81ebbe923667305b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45397 } } }
I20260619 12:24:24.861177 8700 tablet_replica.cc:442] T e5f4616899f245e68318cf36397c9348 P 177bd965fc164fa08b200984ca256841: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:24.874415 8688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.870590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:48659 (local address 127.6.231.195:45397)
0619 12:24:24.870718 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.870722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.870973 (+ 251us) server_negotiation.cc:407] Connection header received
0619 12:24:24.871090 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.871093 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.871142 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.871224 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:24.871497 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.872263 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.873192 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.873543 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.873645 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.873932 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:24.874018 (+ 86us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:24.874093 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.874218 (+ 125us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.874266 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:24:24.874415 8740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.870480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:45397 (local address 127.6.231.195:48659)
0619 12:24:24.870893 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:24.870906 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:24.870997 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:24.871286 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:24.871288 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:24.871296 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:24:24.871389 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.871394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.872384 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.872387 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.873078 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:24.873087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.873681 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.873684 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:24.873758 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.873803 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:24:24.874107 (+ 304us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:24:24.874112 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:24:24.874187 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 12:24:24.874254 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":148,"threads_started":1}
I20260619 12:24:24.881259 8484 raft_consensus.cc:1261] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841 [term 1 FOLLOWER]: Refusing update from remote peer 809ea9dfdc2749ce81ebbe923667305b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:24.881605 8344 raft_consensus.cc:1261] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb [term 1 FOLLOWER]: Refusing update from remote peer 809ea9dfdc2749ce81ebbe923667305b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:24.881757 8701 consensus_queue.cc:1035] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [LEADER]: Connected to new peer: Peer: permanent_uuid: "177bd965fc164fa08b200984ca256841" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34845 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:24.882043 8701 consensus_queue.cc:1035] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6908a8bc11f4f71b171b9c90de479eb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 43263 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:24.886869 8706 mvcc.cc:205] Tried to move back new op lower bound from 7298547158550368256 to 7298547157705179136. Current Snapshot: MvccSnapshot[applied={T|T < 7298547158550368256}]
I20260619 12:24:24.919893 8187 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:24.920085 8187 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:24.922248 8187 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 96241ec541734694a20b930cc398ef1d: Sending DeleteTablet for 3 replicas of tablet 97da2de37080440da8358f0b6b46d6fa
I20260619 12:24:24.922959 8324 tablet_service.cc:1514] Processing DeleteTablet for tablet 97da2de37080440da8358f0b6b46d6fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:24 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:54878
I20260619 12:24:24.922973 8464 tablet_service.cc:1514] Processing DeleteTablet for tablet 97da2de37080440da8358f0b6b46d6fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:24 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:47754
I20260619 12:24:24.923216 8716 tablet_replica.cc:330] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841: stopping tablet replica
I20260619 12:24:24.923337 8716 raft_consensus.cc:2229] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:24.923462 8716 raft_consensus.cc:2258] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:24.924079 8718 tablet_replica.cc:330] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb: stopping tablet replica
I20260619 12:24:24.924222 8718 raft_consensus.cc:2229] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:24.924352 8718 raft_consensus.cc:2258] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:24.925243 8604 tablet_service.cc:1514] Processing DeleteTablet for tablet 97da2de37080440da8358f0b6b46d6fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:24 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:50426
I20260619 12:24:24.925383 8716 ts_tablet_manager.cc:1905] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:24.925487 8717 tablet_replica.cc:330] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b: stopping tablet replica
I20260619 12:24:24.925628 8717 raft_consensus.cc:2229] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:24:24.925834 8717 raft_consensus.cc:2258] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:24.926573 8717 ts_tablet_manager.cc:1905] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:24.928508 8716 ts_tablet_manager.cc:1918] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:24:24.928576 8716 log.cc:1196] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/97da2de37080440da8358f0b6b46d6fa
I20260619 12:24:24.928836 8716 ts_tablet_manager.cc:1939] T 97da2de37080440da8358f0b6b46d6fa P 177bd965fc164fa08b200984ca256841: Deleting consensus metadata
I20260619 12:24:24.929363 8717 ts_tablet_manager.cc:1918] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:24:24.929477 8717 log.cc:1196] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/97da2de37080440da8358f0b6b46d6fa
I20260619 12:24:24.929781 8717 ts_tablet_manager.cc:1939] T 97da2de37080440da8358f0b6b46d6fa P 809ea9dfdc2749ce81ebbe923667305b: Deleting consensus metadata
I20260619 12:24:24.930228 8173 catalog_manager.cc:4841] TS 177bd965fc164fa08b200984ca256841 (127.6.231.194:34845): tablet 97da2de37080440da8358f0b6b46d6fa (table test-table [id=dc979f3bea0c4aab94461afe12619df7]) successfully deleted
I20260619 12:24:24.930778 8173 catalog_manager.cc:4841] TS 809ea9dfdc2749ce81ebbe923667305b (127.6.231.195:45397): tablet 97da2de37080440da8358f0b6b46d6fa (table test-table [id=dc979f3bea0c4aab94461afe12619df7]) successfully deleted
I20260619 12:24:24.931267 8718 ts_tablet_manager.cc:1905] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb: Deleting tablet data with delete state TABLET_DATA_DELETED
Jun 19 12:24:24 dist-test-slave-kr9h krb5kdc[8142](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871864, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
I20260619 12:24:24.934278 8718 ts_tablet_manager.cc:1918] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:24:24.934379 8718 log.cc:1196] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1781871857195028-7071-0/minicluster-data/ts-0/wal/wals/97da2de37080440da8358f0b6b46d6fa
I20260619 12:24:24.934623 8718 ts_tablet_manager.cc:1939] T 97da2de37080440da8358f0b6b46d6fa P a6908a8bc11f4f71b171b9c90de479eb: Deleting consensus metadata
I20260619 12:24:24.935422 8171 catalog_manager.cc:4841] TS a6908a8bc11f4f71b171b9c90de479eb (127.6.231.193:43263): tablet 97da2de37080440da8358f0b6b46d6fa (table test-table [id=dc979f3bea0c4aab94461afe12619df7]) successfully deleted
I20260619 12:24:24.945051 8691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.924629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54886 (local address 127.6.231.193:43263)
0619 12:24:24.924757 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.924763 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.924781 (+ 18us) server_negotiation.cc:407] Connection header received
0619 12:24:24.924823 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.924826 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.924872 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.924958 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:24.926196 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.926919 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.928175 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.928494 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.928591 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.932678 (+ 4087us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:24.932691 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:24.932707 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:24.932737 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:24.942996 (+ 10259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.943589 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.943596 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.943598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.943704 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.944017 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.944021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.944023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.944301 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:24.944560 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.944741 (+ 181us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.944880 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260619 12:24:24.945900 8384 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54886
I20260619 12:24:24.956636 8236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:24.947804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54710 (local address 127.6.231.254:41385)
0619 12:24:24.947940 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:24.947945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:24.947961 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:24:24.948074 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:24.948078 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:24.948125 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:24.948211 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:24.948965 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.949821 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.950720 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:24.951054 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:24.951152 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:24.952732 (+ 1580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:24.952736 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:24.952740 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:24.952772 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:24.954641 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.955241 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.955245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.955247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.955347 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:24.955682 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:24.955686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:24.955688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:24.955930 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:24.956118 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:24.956338 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 12:24:24.956470 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 12:24:24.957109 8187 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54710
I20260619 12:24:24.959280 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 8259
I20260619 12:24:24.966487 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 8399
I20260619 12:24:24.973713 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 8539
I20260619 12:24:24.980718 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 8155
2026-06-19T12:24:24Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2886 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:24:27 dist-test-slave-kr9h krb5kdc[8750](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:24:27 dist-test-slave-kr9h krb5kdc[8750](info): set up 2 sockets
Jun 19 12:24:27 dist-test-slave-kr9h krb5kdc[8750](info): commencing operation
krb5kdc: starting...
W20260619 12:24:28.641752 7071 mini_kdc.cc:121] Time spent starting KDC: real 3.636s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:24:28 dist-test-slave-kr9h krb5kdc[8750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:24:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:24:28.767913 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:39203
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:39203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:28.879501 8763 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:28.879776 8763 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:28.879817 8763 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:28.883882 8763 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:28.883944 8763 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 12:24:28.883960 8763 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:28.883980 8763 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:28.884215 8763 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:28.884292 8763 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:28.889855 8763 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:39203
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:39203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8763
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:28.890969 8763 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:28.892014 8763 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:28.900849 8768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:28.900971 8763 server_base.cc:1034] running on GCE node
W20260619 12:24:28.900920 8769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:28.900913 8771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:28.901978 8763 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:28.903112 8763 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:28.904330 8763 hybrid_clock.cc:648] HybridClock initialized: now 1781871868904286 us; error 55 us; skew 500 ppm
Jun 19 12:24:28 dist-test-slave-kr9h krb5kdc[8750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:28.911334 8763 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:24:28.912370 8763 webserver.cc:460] Webserver started at http://127.6.231.254:34901/ using document root <none> and password file <none>
I20260619 12:24:28.912984 8763 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:28.913048 8763 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:28.913291 8763 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:28.915472 8763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "d9b8e19186fc463194cf3d594fb03591"
format_stamp: "Formatted at 2026-06-19 12:24:28 on dist-test-slave-kr9h"
server_key: "5cb06f806b15885887ec91830bdd1553"
server_key_iv: "ce0916c196e6f9e5ead119ca96530ce6"
server_key_version: "encryptionkey@0"
I20260619 12:24:28.916055 8763 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "d9b8e19186fc463194cf3d594fb03591"
format_stamp: "Formatted at 2026-06-19 12:24:28 on dist-test-slave-kr9h"
server_key: "5cb06f806b15885887ec91830bdd1553"
server_key_iv: "ce0916c196e6f9e5ead119ca96530ce6"
server_key_version: "encryptionkey@0"
I20260619 12:24:28.919937 8763 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:24:28.922588 8778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:28.923964 8763 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:28.924114 8763 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "d9b8e19186fc463194cf3d594fb03591"
format_stamp: "Formatted at 2026-06-19 12:24:28 on dist-test-slave-kr9h"
server_key: "5cb06f806b15885887ec91830bdd1553"
server_key_iv: "ce0916c196e6f9e5ead119ca96530ce6"
server_key_version: "encryptionkey@0"
I20260619 12:24:28.924224 8763 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:28.945472 8763 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:28.948709 8763 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:28.948921 8763 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:28.957170 8763 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:39203
I20260619 12:24:28.957187 8840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:39203 every 8 connection(s)
I20260619 12:24:28.958211 8763 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:28.961004 8841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:28.964526 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 8763
I20260619 12:24:28.964643 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:28.964915 7071 external_mini_cluster.cc:1428] Setting key 769a45aa413fa272adc6bba921f73f79
I20260619 12:24:28.967082 8841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591: Bootstrap starting.
I20260619 12:24:28.969223 8841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:28.970000 8841 log.cc:824] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:28.972170 8841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591: No bootstrap required, opened a new log
Jun 19 12:24:28 dist-test-slave-kr9h krb5kdc[8750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871868, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:28.975955 8841 raft_consensus.cc:348] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b8e19186fc463194cf3d594fb03591" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 39203 } }
I20260619 12:24:28.976171 8841 raft_consensus.cc:374] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:28.976320 8841 raft_consensus.cc:729] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d9b8e19186fc463194cf3d594fb03591, State: Initialized, Role: FOLLOWER
I20260619 12:24:28.976861 8841 consensus_queue.cc:260] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b8e19186fc463194cf3d594fb03591" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 39203 } }
I20260619 12:24:28.977030 8841 raft_consensus.cc:388] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:28.977103 8841 raft_consensus.cc:482] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:28.977208 8841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:28.978399 8841 raft_consensus.cc:504] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b8e19186fc463194cf3d594fb03591" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 39203 } }
I20260619 12:24:28.978767 8841 leader_election.cc:302] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d9b8e19186fc463194cf3d594fb03591; no voters:
I20260619 12:24:28.979158 8841 leader_election.cc:288] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:28.979380 8846 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:28.979813 8846 raft_consensus.cc:686] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [term 1 LEADER]: Becoming Leader. State: Replica: d9b8e19186fc463194cf3d594fb03591, State: Running, Role: LEADER
I20260619 12:24:28.980209 8846 consensus_queue.cc:237] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b8e19186fc463194cf3d594fb03591" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 39203 } }
I20260619 12:24:28.980541 8841 sys_catalog.cc:564] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:28.980690 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:28.966478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37990 (local address 127.6.231.254:39203)
0619 12:24:28.966917 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:28.966932 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:28.966967 (+ 35us) server_negotiation.cc:407] Connection header received
0619 12:24:28.967684 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:28.967703 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:28.968037 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:28.968386 (+ 349us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:28.969167 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:28.970328 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:28.971546 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:28.972004 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:28.972130 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:28.975088 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:28.975093 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:28.975107 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:28.975135 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:28.977319 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:28.977841 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:28.977848 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:28.977853 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:28.977993 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:28.978322 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:28.978328 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:28.978333 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:28.978850 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:28.979039 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:28.979388 (+ 349us) server_negotiation.cc:299] Negotiation successful
0619 12:24:28.979693 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":134,"threads_started":1}
I20260619 12:24:28.981554 8846 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d9b8e19186fc463194cf3d594fb03591. Latest consensus state: current_term: 1 leader_uuid: "d9b8e19186fc463194cf3d594fb03591" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b8e19186fc463194cf3d594fb03591" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 39203 } } }
I20260619 12:24:28.981779 8846 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:28.981990 8848 sys_catalog.cc:455] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d9b8e19186fc463194cf3d594fb03591" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d9b8e19186fc463194cf3d594fb03591" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 39203 } } }
I20260619 12:24:28.982088 8848 sys_catalog.cc:458] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591 [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:28.983019 8855 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:28.985831 8855 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:28.991638 8855 catalog_manager.cc:1347] Generated new cluster ID: e54ebea215a145d9b8c7778217419bb7
I20260619 12:24:28.991787 8855 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:29.000787 8855 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:29.002202 8855 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:29.012050 8855 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591: Generated new TSK 0
I20260619 12:24:29.012871 8855 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:24:29.040943 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:39203
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:29.156096 8867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:29.156512 8867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:29.156564 8867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:29.160702 8867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:29.160774 8867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:29.160818 8867 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 12:24:29.160833 8867 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:24:29.160915 8867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:24:29.166574 8867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.231.254:39203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:29.167685 8867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:29.168731 8867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:29.178719 8873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:29.178689 8872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:29.178699 8875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:29.179085 8867 server_base.cc:1034] running on GCE node
I20260619 12:24:29.179545 8867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:29.180250 8867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:29.181414 8867 hybrid_clock.cc:648] HybridClock initialized: now 1781871869181393 us; error 32 us; skew 500 ppm
Jun 19 12:24:29 dist-test-slave-kr9h krb5kdc[8750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871869, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:29.184391 8867 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:24:29.185288 8867 webserver.cc:460] Webserver started at http://127.6.231.193:41493/ using document root <none> and password file <none>
I20260619 12:24:29.185901 8867 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:29.185992 8867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:29.186213 8867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:29.188269 8867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "f6de249802464d9481b9af89b88ce390"
format_stamp: "Formatted at 2026-06-19 12:24:29 on dist-test-slave-kr9h"
server_key: "394987188ff6772a74440e6df86cb007"
server_key_iv: "4133794ac3ac938b1313bea64f798979"
server_key_version: "encryptionkey@0"
I20260619 12:24:29.188838 8867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "f6de249802464d9481b9af89b88ce390"
format_stamp: "Formatted at 2026-06-19 12:24:29 on dist-test-slave-kr9h"
server_key: "394987188ff6772a74440e6df86cb007"
server_key_iv: "4133794ac3ac938b1313bea64f798979"
server_key_version: "encryptionkey@0"
I20260619 12:24:29.192516 8867 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260619 12:24:29.195039 8882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:29.196333 8867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:29.196471 8867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "f6de249802464d9481b9af89b88ce390"
format_stamp: "Formatted at 2026-06-19 12:24:29 on dist-test-slave-kr9h"
server_key: "394987188ff6772a74440e6df86cb007"
server_key_iv: "4133794ac3ac938b1313bea64f798979"
server_key_version: "encryptionkey@0"
I20260619 12:24:29.196594 8867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:29.237821 8867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:29.242303 8867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:29.242525 8867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:29.244529 8867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:29.244603 8867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:29.244675 8867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:29.244717 8867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:24:29 dist-test-slave-kr9h krb5kdc[8750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871869, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:29.258256 8997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:41685 every 8 connection(s)
I20260619 12:24:29.258303 8867 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:41685
I20260619 12:24:29.259687 8867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:24:29.260201 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 8867
I20260619 12:24:29.260339 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:24:29.260668 7071 external_mini_cluster.cc:1428] Setting key 1363ad32a5dc5d005e6e2447d2469a2d
I20260619 12:24:29.262773 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.245812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:38289 (local address 127.6.231.254:39203)
0619 12:24:29.246797 (+ 985us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.246805 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.247361 (+ 556us) server_negotiation.cc:407] Connection header received
0619 12:24:29.248896 (+ 1535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.248900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.248972 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.249079 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:29.250680 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.251596 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.253563 (+ 1967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.253968 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.254080 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.257897 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:29.257901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:29.257905 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:29.257936 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:29.260007 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.260714 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.260718 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.260720 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.260824 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.261355 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.261358 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.261362 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.261629 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:29.261727 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.262457 (+ 730us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.262596 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":872}
I20260619 12:24:29.263851 8891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.246405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:39203 (local address 127.6.231.193:38289)
0619 12:24:29.247186 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.247247 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.248569 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.249280 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.249288 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.249809 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:29.250520 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.250541 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.251726 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.251731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.253356 (+ 1625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.253368 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.254477 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.254481 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.254705 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.255452 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:29.255473 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:29.257640 (+ 2167us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:29.260158 (+ 2518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.260166 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.260182 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.260567 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.260933 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.260936 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.260938 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.261143 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.261754 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:29.261761 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:29.262093 (+ 332us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.262386 (+ 293us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.262753 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":358,"threads_started":1}
I20260619 12:24:29.265306 8998 heartbeater.cc:347] Connected to a master server at 127.6.231.254:39203
I20260619 12:24:29.265475 8998 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:29.266050 8998 heartbeater.cc:511] Master 127.6.231.254:39203 requested a full tablet report, sending...
I20260619 12:24:29.267750 8794 ts_manager.cc:194] Registered new tserver with Master: f6de249802464d9481b9af89b88ce390 (127.6.231.193:41685)
I20260619 12:24:29.269155 8794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:38289
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:24:29.275872 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.267858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:53343 (local address 127.6.231.254:39203)
0619 12:24:29.267963 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.267967 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.267980 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:24:29.268045 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.268048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.268089 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.268170 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:29.268737 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.269501 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.271051 (+ 1550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.271454 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.271573 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.272787 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:29.272794 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:29.272799 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:29.272832 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:29.274223 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.274593 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.274596 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.274597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.274667 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.274961 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.274964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.274965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.275267 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:29.275354 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.275567 (+ 213us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.275682 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 12:24:29.275900 9005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.267530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:39203 (local address 127.6.231.193:53343)
0619 12:24:29.267840 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.267857 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.267952 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.268208 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.268214 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.268461 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:29.268603 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.268611 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.269648 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.269651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.270915 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.270925 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.271588 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.271592 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.271680 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.272269 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:29.272317 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:29.272643 (+ 326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:29.274318 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.274320 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.274322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.274513 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.274778 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.274780 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.274781 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.274854 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.275360 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:29.275363 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:29.275438 (+ 75us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.275535 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.275644 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":93,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:24:29.292164 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:39203
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:29.402669 9007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:29.402979 9007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:29.403057 9007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:29.407114 9007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:29.407215 9007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:29.407289 9007 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 12:24:29.407330 9007 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:24:29.407398 9007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:24:29.412776 9007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.231.254:39203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:29.414003 9007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:29.415071 9007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:29.425199 9013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:29.425190 9012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:29.425251 9015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:29.425483 9007 server_base.cc:1034] running on GCE node
I20260619 12:24:29.425944 9007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:29.426578 9007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:29.427789 9007 hybrid_clock.cc:648] HybridClock initialized: now 1781871869427763 us; error 32 us; skew 500 ppm
Jun 19 12:24:29 dist-test-slave-kr9h krb5kdc[8750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871869, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:29.430817 9007 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:24:29.431665 9007 webserver.cc:460] Webserver started at http://127.6.231.194:37225/ using document root <none> and password file <none>
I20260619 12:24:29.432188 9007 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:29.432256 9007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:29.432420 9007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:29.434525 9007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "3002b4681a3b4337ba94667b62a849be"
format_stamp: "Formatted at 2026-06-19 12:24:29 on dist-test-slave-kr9h"
server_key: "7fa272cf42dbc7128da9e061e8d26471"
server_key_iv: "529ac8135182a601d0725543a3cb5903"
server_key_version: "encryptionkey@0"
I20260619 12:24:29.435036 9007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "3002b4681a3b4337ba94667b62a849be"
format_stamp: "Formatted at 2026-06-19 12:24:29 on dist-test-slave-kr9h"
server_key: "7fa272cf42dbc7128da9e061e8d26471"
server_key_iv: "529ac8135182a601d0725543a3cb5903"
server_key_version: "encryptionkey@0"
I20260619 12:24:29.438871 9007 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:24:29.441329 9022 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:29.442692 9007 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:24:29.442792 9007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "3002b4681a3b4337ba94667b62a849be"
format_stamp: "Formatted at 2026-06-19 12:24:29 on dist-test-slave-kr9h"
server_key: "7fa272cf42dbc7128da9e061e8d26471"
server_key_iv: "529ac8135182a601d0725543a3cb5903"
server_key_version: "encryptionkey@0"
I20260619 12:24:29.442883 9007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:29.456241 9007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:29.459595 9007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:29.459789 9007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:29.461274 9007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:29.461323 9007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:29.461364 9007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:29.461418 9007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:24:29 dist-test-slave-kr9h krb5kdc[8750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871869, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:29.472003 9007 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:39721
I20260619 12:24:29.472065 9137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:39721 every 8 connection(s)
I20260619 12:24:29.472898 9007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:24:29.475484 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.462204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:53475 (local address 127.6.231.254:39203)
0619 12:24:29.462351 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.462356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.463290 (+ 934us) server_negotiation.cc:407] Connection header received
0619 12:24:29.464322 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.464326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.464384 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.464486 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:29.465945 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.466669 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.467924 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.468325 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.468464 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.471891 (+ 3427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:29.471894 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:29.471896 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:29.471921 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:29.473534 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.474098 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.474101 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.474103 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.474180 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.474480 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.474482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.474484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.474654 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:29.474732 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.475265 (+ 533us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.475365 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 12:24:29.476393 9031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.462673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:39203 (local address 127.6.231.194:53475)
0619 12:24:29.463123 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.463174 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.464102 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.464685 (+ 583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.464694 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.465123 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:29.465794 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.465812 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.466794 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.466798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.467806 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.467816 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.468513 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.468519 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.468750 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.469971 (+ 1221us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:29.469986 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:29.471665 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:29.473660 (+ 1995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.473665 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.473679 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.474006 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.474262 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.474264 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.474265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.474404 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.474775 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:29.474781 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:29.475003 (+ 222us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.475214 (+ 211us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.475461 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":120,"threads_started":1}
I20260619 12:24:29.477706 9138 heartbeater.cc:347] Connected to a master server at 127.6.231.254:39203
I20260619 12:24:29.477873 9138 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:29.478438 9138 heartbeater.cc:511] Master 127.6.231.254:39203 requested a full tablet report, sending...
I20260619 12:24:29.478965 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 9007
I20260619 12:24:29.479085 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:24:29.479393 7071 external_mini_cluster.cc:1428] Setting key 558858e568f1ed38a783ca4bc2f84e5b
I20260619 12:24:29.479507 8795 ts_manager.cc:194] Registered new tserver with Master: 3002b4681a3b4337ba94667b62a849be (127.6.231.194:39721)
I20260619 12:24:29.480358 8795 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:53475
I20260619 12:24:29.488642 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.480239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:57823 (local address 127.6.231.254:39203)
0619 12:24:29.480342 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.480346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.480361 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:24:29.480514 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.480517 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.480559 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.480636 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:29.481235 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.482077 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.483411 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.483789 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.483909 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.485203 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:29.485206 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:29.485209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:29.485231 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:29.486801 (+ 1570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.487342 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.487345 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.487346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.487424 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.487770 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.487773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.487774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.488006 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:29.488088 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.488414 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.488521 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:24:29.488847 9144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.479807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:39203 (local address 127.6.231.194:57823)
0619 12:24:29.480278 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.480293 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.480383 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.480653 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.480656 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.480958 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:29.481084 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.481092 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.482553 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.482559 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.483295 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.483304 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.483931 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.483934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.484010 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.484574 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:29.484593 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:29.485063 (+ 470us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:29.486916 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.486921 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.486927 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.487216 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.487525 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.487530 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.487534 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.487652 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.488107 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:29.488110 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:29.488214 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.488343 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.488510 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:24:29.508819 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:39203
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:42033
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:29.622581 9147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:29.622853 9147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:29.622901 9147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:29.627185 9147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:29.627259 9147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:29.627301 9147 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260619 12:24:29.627316 9147 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:24:29.627349 9147 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:24:29.632735 9147 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:42033
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.231.254:39203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:29.633910 9147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:29.634970 9147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:29.644459 9153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:29.644542 9155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:29.644459 9152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:29.644553 9147 server_base.cc:1034] running on GCE node
I20260619 12:24:29.645208 9147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:29.645982 9147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:29.647167 9147 hybrid_clock.cc:648] HybridClock initialized: now 1781871869647152 us; error 35 us; skew 500 ppm
Jun 19 12:24:29 dist-test-slave-kr9h krb5kdc[8750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871869, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:29.650003 9147 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:24:29.650823 9147 webserver.cc:460] Webserver started at http://127.6.231.195:33073/ using document root <none> and password file <none>
I20260619 12:24:29.651346 9147 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:29.651403 9147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:29.651562 9147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:29.653581 9147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c"
format_stamp: "Formatted at 2026-06-19 12:24:29 on dist-test-slave-kr9h"
server_key: "9d5e93c4e5fbdb65bcb0a613f5634331"
server_key_iv: "b526b1bd58fc33e4672e55461e7f50d7"
server_key_version: "encryptionkey@0"
I20260619 12:24:29.654102 9147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c"
format_stamp: "Formatted at 2026-06-19 12:24:29 on dist-test-slave-kr9h"
server_key: "9d5e93c4e5fbdb65bcb0a613f5634331"
server_key_iv: "b526b1bd58fc33e4672e55461e7f50d7"
server_key_version: "encryptionkey@0"
I20260619 12:24:29.657781 9147 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:29.660081 9162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:29.661399 9147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:24:29.661592 9147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c"
format_stamp: "Formatted at 2026-06-19 12:24:29 on dist-test-slave-kr9h"
server_key: "9d5e93c4e5fbdb65bcb0a613f5634331"
server_key_iv: "b526b1bd58fc33e4672e55461e7f50d7"
server_key_version: "encryptionkey@0"
I20260619 12:24:29.661729 9147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:29.684118 9147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:29.687971 9147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:29.688179 9147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:29.689718 9147 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:29.689790 9147 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:29.689863 9147 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:29.689903 9147 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:24:29 dist-test-slave-kr9h krb5kdc[8750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871869, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:29.700920 9147 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:39673
I20260619 12:24:29.700953 9277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:39673 every 8 connection(s)
I20260619 12:24:29.702414 9147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:24:29.705098 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.690621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:44851 (local address 127.6.231.254:39203)
0619 12:24:29.690765 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.690769 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.692072 (+ 1303us) server_negotiation.cc:407] Connection header received
0619 12:24:29.693139 (+ 1067us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.693143 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.693201 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.693322 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:29.694986 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.695587 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.697488 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.697856 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.697977 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.701200 (+ 3223us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:29.701204 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:29.701207 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:29.701234 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:29.702991 (+ 1757us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.703638 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.703641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.703642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.703737 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.704061 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.704064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.704065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.704280 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:29.704366 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.704856 (+ 490us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.704967 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:24:29.705641 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 9147
I20260619 12:24:29.705740 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:24:29.706015 7071 external_mini_cluster.cc:1428] Setting key b774b9eecfd1f14f969a8c39df49691b
I20260619 12:24:29.706229 9171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.691411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:39203 (local address 127.6.231.195:44851)
0619 12:24:29.691923 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.691967 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.692912 (+ 945us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.693566 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.693576 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.694059 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:29.694817 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.694839 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.695700 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.695705 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.697299 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.697314 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.698013 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.698017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.698235 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.698858 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:29.698888 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:29.700985 (+ 2097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:29.703164 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.703173 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.703192 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.703499 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.703830 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.703832 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.703833 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.703979 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.704379 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:29.704384 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:29.704624 (+ 240us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.704793 (+ 169us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.705043 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":129,"threads_started":1}
I20260619 12:24:29.707518 9278 heartbeater.cc:347] Connected to a master server at 127.6.231.254:39203
I20260619 12:24:29.707659 9278 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:29.708177 9278 heartbeater.cc:511] Master 127.6.231.254:39203 requested a full tablet report, sending...
I20260619 12:24:29.709118 8794 ts_manager.cc:194] Registered new tserver with Master: 4ade6c8d99ca4b6aa9ce40101ba2a99c (127.6.231.195:39673)
I20260619 12:24:29.709771 8794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:44851
I20260619 12:24:29.717020 9284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.709930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:39203 (local address 127.6.231.195:47349)
0619 12:24:29.710420 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.710510 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.710616 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.710854 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.710857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.711070 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:29.711175 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.711182 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.711941 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.711944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.712488 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.712495 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.712935 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.712938 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.713016 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.713598 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:29.713618 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:29.713955 (+ 337us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:29.715600 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.715603 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.715605 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.715831 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.716072 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.716075 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.716076 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.716172 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.716564 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:29.716567 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:29.716672 (+ 105us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.716775 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.716884 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":92,"threads_started":1}
I20260619 12:24:29.717026 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.709970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:47349 (local address 127.6.231.254:39203)
0619 12:24:29.710091 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.710096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.710620 (+ 524us) server_negotiation.cc:407] Connection header received
0619 12:24:29.710712 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.710714 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.710750 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.710818 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:29.711308 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.711828 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.712586 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.712817 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.712898 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.714079 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:29.714082 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:29.714084 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:29.714105 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:29.715477 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.715925 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.715928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.715930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.715997 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.716279 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.716282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.716284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.716455 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:29.716539 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.716828 (+ 289us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.716920 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:24:29.720253 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:24:29 dist-test-slave-kr9h krb5kdc[8750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:24:29 dist-test-slave-kr9h krb5kdc[8750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871869, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:29.746358 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.738112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56778 (local address 127.6.231.254:39203)
0619 12:24:29.738298 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.738302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.738315 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:24:29.738360 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.738362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.738401 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.738479 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:29.739166 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.739731 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.740532 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.740745 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.740824 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.743348 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:29.743351 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:29.743354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:29.743378 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:29.744629 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.745047 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.745052 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.745054 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.745132 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.745416 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.745419 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.745420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.745647 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:29.745804 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.746056 (+ 252us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.746183 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:24:29.749607 8794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:24:29.752761 8794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:29.767856 9298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.761816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51612 (local address 127.6.231.194:39721)
0619 12:24:29.762141 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.762157 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.762313 (+ 156us) server_negotiation.cc:407] Connection header received
0619 12:24:29.762451 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.762455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.763066 (+ 611us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.763490 (+ 424us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:29.763615 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.764666 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.766291 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.767112 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.767261 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.767417 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.767605 (+ 188us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.767685 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":113,"threads_started":1}
I20260619 12:24:29.767899 9296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.761368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:39721 (local address 127.0.0.1:51612)
0619 12:24:29.762207 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.762224 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.762333 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.763239 (+ 906us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.763242 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.763259 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:29.763356 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.763362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.764792 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.764796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.766169 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.766178 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.767260 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.767265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.767409 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.767449 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.767575 (+ 126us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.767718 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":769,"mutex_wait_us":70,"thread_start_us":146,"threads_started":1}
I20260619 12:24:29.768281 9300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.762610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37568 (local address 127.6.231.195:39673)
0619 12:24:29.763099 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.763113 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.763137 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:24:29.763216 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.763221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.763405 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.763565 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:29.763818 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.764852 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.766647 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.767282 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.767397 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.767550 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.767850 (+ 300us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.768113 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":128,"threads_started":1}
I20260619 12:24:29.768546 9299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.762178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:39673 (local address 127.0.0.1:37568)
0619 12:24:29.762712 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.762731 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.762825 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.763552 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.763555 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.763580 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:29.763700 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.763713 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.765055 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.765061 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.766526 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.766534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.767615 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.767619 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.767704 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.767725 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.767808 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.767874 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":93,"threads_started":1}
I20260619 12:24:29.768591 9295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.761247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:41685 (local address 127.0.0.1:48514)
0619 12:24:29.761592 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.761647 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.761775 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.762743 (+ 968us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.762749 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.762774 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:29.762918 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.762925 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.764576 (+ 1651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.764580 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.766320 (+ 1740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.766331 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.767288 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.767291 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.768274 (+ 983us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.768299 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.768373 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.768453 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":884,"thread_start_us":81,"threads_started":1}
I20260619 12:24:29.769109 9297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.761548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48514 (local address 127.6.231.193:41685)
0619 12:24:29.762167 (+ 619us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.762181 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.762203 (+ 22us) server_negotiation.cc:407] Connection header received
0619 12:24:29.762293 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.762299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.762524 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.763096 (+ 572us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:29.763250 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.764448 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.766440 (+ 1992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.767156 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.767379 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.767496 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.768864 (+ 1368us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.768975 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":91,"threads_started":1}
I20260619 12:24:29.771170 9211 tablet_service.cc:1467] Processing CreateTablet for tablet f9d39c6ef701471d818c82a91a8b4dd8 (DEFAULT_TABLE table=test-table [id=6a6959fa34ec4d51a75952e81893b88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:29.771170 9072 tablet_service.cc:1467] Processing CreateTablet for tablet f9d39c6ef701471d818c82a91a8b4dd8 (DEFAULT_TABLE table=test-table [id=6a6959fa34ec4d51a75952e81893b88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:29.771569 8930 tablet_service.cc:1467] Processing CreateTablet for tablet f9d39c6ef701471d818c82a91a8b4dd8 (DEFAULT_TABLE table=test-table [id=6a6959fa34ec4d51a75952e81893b88b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:29.772207 9211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9d39c6ef701471d818c82a91a8b4dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:29.772220 9072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9d39c6ef701471d818c82a91a8b4dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:29.772413 8930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9d39c6ef701471d818c82a91a8b4dd8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:29.780244 9301 tablet_bootstrap.cc:492] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Bootstrap starting.
I20260619 12:24:29.780920 9303 tablet_bootstrap.cc:492] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390: Bootstrap starting.
I20260619 12:24:29.781594 9302 tablet_bootstrap.cc:492] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be: Bootstrap starting.
I20260619 12:24:29.782683 9301 tablet_bootstrap.cc:654] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:29.783066 9303 tablet_bootstrap.cc:654] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:29.783557 9301 log.cc:824] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:29.783735 9302 tablet_bootstrap.cc:654] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:29.784341 9303 log.cc:824] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:29.784552 9302 log.cc:824] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:29.786006 9301 tablet_bootstrap.cc:492] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: No bootstrap required, opened a new log
I20260619 12:24:29.786230 9301 ts_tablet_manager.cc:1397] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260619 12:24:29.786263 9303 tablet_bootstrap.cc:492] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390: No bootstrap required, opened a new log
I20260619 12:24:29.786453 9303 ts_tablet_manager.cc:1397] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 12:24:29.787120 9302 tablet_bootstrap.cc:492] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be: No bootstrap required, opened a new log
I20260619 12:24:29.787307 9302 ts_tablet_manager.cc:1397] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260619 12:24:29.789327 9303 raft_consensus.cc:348] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.789619 9303 raft_consensus.cc:374] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:29.789674 9303 raft_consensus.cc:729] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6de249802464d9481b9af89b88ce390, State: Initialized, Role: FOLLOWER
I20260619 12:24:29.790105 9303 consensus_queue.cc:260] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.790776 8998 heartbeater.cc:503] Master 127.6.231.254:39203 was elected leader, sending a full tablet report...
I20260619 12:24:29.790762 9301 raft_consensus.cc:348] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.791008 9301 raft_consensus.cc:374] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:29.791061 9301 raft_consensus.cc:729] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ade6c8d99ca4b6aa9ce40101ba2a99c, State: Initialized, Role: FOLLOWER
I20260619 12:24:29.791134 9303 ts_tablet_manager.cc:1428] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 12:24:29.791509 9301 consensus_queue.cc:260] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.791615 9302 raft_consensus.cc:348] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.791875 9302 raft_consensus.cc:374] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:29.791926 9302 raft_consensus.cc:729] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3002b4681a3b4337ba94667b62a849be, State: Initialized, Role: FOLLOWER
I20260619 12:24:29.792356 9302 consensus_queue.cc:260] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.792629 9301 ts_tablet_manager.cc:1428] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 12:24:29.793007 9278 heartbeater.cc:503] Master 127.6.231.254:39203 was elected leader, sending a full tablet report...
I20260619 12:24:29.793320 9302 ts_tablet_manager.cc:1428] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 12:24:29.793706 9138 heartbeater.cc:503] Master 127.6.231.254:39203 was elected leader, sending a full tablet report...
I20260619 12:24:29.814728 9307 raft_consensus.cc:482] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:29.814968 9307 raft_consensus.cc:504] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.816119 9307 leader_election.cc:288] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3002b4681a3b4337ba94667b62a849be (127.6.231.194:39721), 4ade6c8d99ca4b6aa9ce40101ba2a99c (127.6.231.195:39673)
I20260619 12:24:29.820112 9298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.816525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:45107 (local address 127.6.231.194:39721)
0619 12:24:29.816654 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.816658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.816818 (+ 160us) server_negotiation.cc:407] Connection header received
0619 12:24:29.816980 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.816983 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.817035 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.817139 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:29.817478 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.818004 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.819140 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.819560 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.819689 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.819755 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.819918 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.819985 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:24:29.820139 9310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.816391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:39721 (local address 127.6.231.193:45107)
0619 12:24:29.816682 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.816736 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.816852 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.817155 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.817159 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.817177 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:29.817336 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.817342 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.818114 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.818120 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.819003 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.819012 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.819677 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.819680 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.819785 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.819808 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.819900 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.819976 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":24,"thread_start_us":126,"threads_started":1}
I20260619 12:24:29.820392 9300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.816945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:37191 (local address 127.6.231.195:39673)
0619 12:24:29.817079 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.817083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.817321 (+ 238us) server_negotiation.cc:407] Connection header received
0619 12:24:29.817482 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.817484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.817523 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.817602 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:29.817858 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.818354 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.819413 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.819820 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.819939 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.819986 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.820164 (+ 178us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.820263 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260619 12:24:29.820557 9311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.816824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:39673 (local address 127.6.231.193:37191)
0619 12:24:29.817218 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.817232 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.817349 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.817611 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.817614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.817629 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:29.817754 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.817760 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.818466 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.818471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.819292 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.819301 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.819939 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.819943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.820051 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.820067 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.820149 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.820346 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"mutex_wait_us":43,"thread_start_us":92,"threads_started":1}
I20260619 12:24:29.820791 9092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9d39c6ef701471d818c82a91a8b4dd8" candidate_uuid: "f6de249802464d9481b9af89b88ce390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3002b4681a3b4337ba94667b62a849be" is_pre_election: true
I20260619 12:24:29.821130 9092 raft_consensus.cc:2454] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6de249802464d9481b9af89b88ce390 in term 0.
I20260619 12:24:29.821269 9216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9d39c6ef701471d818c82a91a8b4dd8" candidate_uuid: "f6de249802464d9481b9af89b88ce390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" is_pre_election: true
I20260619 12:24:29.821593 9216 raft_consensus.cc:2454] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6de249802464d9481b9af89b88ce390 in term 0.
I20260619 12:24:29.821707 8885 leader_election.cc:302] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3002b4681a3b4337ba94667b62a849be, f6de249802464d9481b9af89b88ce390; no voters:
I20260619 12:24:29.821995 9307 raft_consensus.cc:2781] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:29.822084 9307 raft_consensus.cc:482] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:29.822129 9307 raft_consensus.cc:3037] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:29.823154 9307 raft_consensus.cc:504] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.823536 9307 leader_election.cc:288] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [CANDIDATE]: Term 1 election: Requested vote from peers 3002b4681a3b4337ba94667b62a849be (127.6.231.194:39721), 4ade6c8d99ca4b6aa9ce40101ba2a99c (127.6.231.195:39673)
I20260619 12:24:29.823926 9216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9d39c6ef701471d818c82a91a8b4dd8" candidate_uuid: "f6de249802464d9481b9af89b88ce390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c"
I20260619 12:24:29.824074 9216 raft_consensus.cc:3037] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:29.824062 9092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9d39c6ef701471d818c82a91a8b4dd8" candidate_uuid: "f6de249802464d9481b9af89b88ce390" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3002b4681a3b4337ba94667b62a849be"
I20260619 12:24:29.824271 9092 raft_consensus.cc:3037] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:29.825176 9216 raft_consensus.cc:2454] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6de249802464d9481b9af89b88ce390 in term 1.
I20260619 12:24:29.825373 9092 raft_consensus.cc:2454] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6de249802464d9481b9af89b88ce390 in term 1.
I20260619 12:24:29.825519 8885 leader_election.cc:302] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ade6c8d99ca4b6aa9ce40101ba2a99c, f6de249802464d9481b9af89b88ce390; no voters:
I20260619 12:24:29.825688 9307 raft_consensus.cc:2781] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:29.825922 9307 raft_consensus.cc:686] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [term 1 LEADER]: Becoming Leader. State: Replica: f6de249802464d9481b9af89b88ce390, State: Running, Role: LEADER
I20260619 12:24:29.826287 9307 consensus_queue.cc:237] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.829542 8793 catalog_manager.cc:5495] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6de249802464d9481b9af89b88ce390 (127.6.231.193). New cstate: current_term: 1 leader_uuid: "f6de249802464d9481b9af89b88ce390" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } health_report { overall_health: UNKNOWN } } }
Jun 19 12:24:29 dist-test-slave-kr9h krb5kdc[8750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871868, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:29.868264 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.859204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:39203 (local address 127.0.0.1:56790)
0619 12:24:29.859513 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.859528 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.859672 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.859932 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.859935 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.860388 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:29.860482 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.860489 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.861341 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.861345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.861971 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.861978 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.862449 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.862453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.862542 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.863112 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:29.863124 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:29.864636 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:29.866627 (+ 1991us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.866638 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.866644 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.866869 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.867185 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:29.867189 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:29.867191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:29.867277 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:29.867659 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:29.867667 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:29.867779 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.867889 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.868050 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":117,"threads_started":1}
I20260619 12:24:29.868264 8844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.859239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56790 (local address 127.6.231.254:39203)
0619 12:24:29.859417 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.859424 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.859612 (+ 188us) server_negotiation.cc:407] Connection header received
0619 12:24:29.859777 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.859780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.859827 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.859916 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:29.860620 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.861199 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.862091 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.862335 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.862447 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.864820 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:29.864823 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:29.864825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:29.864845 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:29.866500 (+ 1655us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.867004 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.867010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.867012 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.867099 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:29.867361 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:29.867366 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:29.867368 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:29.867539 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:29.867659 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.867941 (+ 282us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.868105 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":68}
I20260619 12:24:29.872426 8793 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:56790:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 12:24:29.873112 8793 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:29.877899 8930 tablet_service.cc:1467] Processing CreateTablet for tablet 32f02cdadfe7425da3b852354a81a587 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf6d7027c8214d859bb7c1366251ab30]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:24:29.878268 9072 tablet_service.cc:1467] Processing CreateTablet for tablet 32f02cdadfe7425da3b852354a81a587 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf6d7027c8214d859bb7c1366251ab30]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:24:29.878283 9211 tablet_service.cc:1467] Processing CreateTablet for tablet 32f02cdadfe7425da3b852354a81a587 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=bf6d7027c8214d859bb7c1366251ab30]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:24:29.878746 8930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32f02cdadfe7425da3b852354a81a587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:29.878808 9072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32f02cdadfe7425da3b852354a81a587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:29.878997 9211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 32f02cdadfe7425da3b852354a81a587. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:29.881649 9301 tablet_bootstrap.cc:492] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Bootstrap starting.
I20260619 12:24:29.881680 9303 tablet_bootstrap.cc:492] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390: Bootstrap starting.
I20260619 12:24:29.881680 9302 tablet_bootstrap.cc:492] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be: Bootstrap starting.
I20260619 12:24:29.882704 9303 tablet_bootstrap.cc:654] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:29.882704 9302 tablet_bootstrap.cc:654] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:29.882831 9301 tablet_bootstrap.cc:654] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:29.884189 9302 tablet_bootstrap.cc:492] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be: No bootstrap required, opened a new log
I20260619 12:24:29.884279 9302 ts_tablet_manager.cc:1397] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:24:29.884397 9303 tablet_bootstrap.cc:492] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390: No bootstrap required, opened a new log
I20260619 12:24:29.884483 9303 ts_tablet_manager.cc:1397] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:24:29.884621 9301 tablet_bootstrap.cc:492] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: No bootstrap required, opened a new log
I20260619 12:24:29.884761 9301 ts_tablet_manager.cc:1397] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:24:29.884771 9302 raft_consensus.cc:348] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.884889 9302 raft_consensus.cc:374] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:29.884938 9302 raft_consensus.cc:729] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3002b4681a3b4337ba94667b62a849be, State: Initialized, Role: FOLLOWER
I20260619 12:24:29.884918 9303 raft_consensus.cc:348] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.885027 9303 raft_consensus.cc:374] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:29.885070 9303 raft_consensus.cc:729] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6de249802464d9481b9af89b88ce390, State: Initialized, Role: FOLLOWER
I20260619 12:24:29.885113 9302 consensus_queue.cc:260] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.885186 9303 consensus_queue.cc:260] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.885280 9301 raft_consensus.cc:348] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.885389 9301 raft_consensus.cc:374] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:29.885401 9302 ts_tablet_manager.cc:1428] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:29.885425 9301 raft_consensus.cc:729] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ade6c8d99ca4b6aa9ce40101ba2a99c, State: Initialized, Role: FOLLOWER
I20260619 12:24:29.885421 9309 tablet_replica.cc:439] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } } }
I20260619 12:24:29.885612 9309 tablet_replica.cc:442] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:29.885597 9301 consensus_queue.cc:260] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.885879 9308 tablet_replica.cc:439] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } } }
I20260619 12:24:29.885406 9307 tablet_replica.cc:439] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } } }
I20260619 12:24:29.886039 9307 tablet_replica.cc:442] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:29.886039 9308 tablet_replica.cc:442] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:29.886248 9303 ts_tablet_manager.cc:1428] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:29.885862 9301 ts_tablet_manager.cc:1428] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
W20260619 12:24:29.953492 9279 tablet.cc:2369] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:24:29.967677 9308 raft_consensus.cc:482] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:29.967842 9308 raft_consensus.cc:504] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.969094 9308 leader_election.cc:288] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6de249802464d9481b9af89b88ce390 (127.6.231.193:41685), 3002b4681a3b4337ba94667b62a849be (127.6.231.194:39721)
I20260619 12:24:29.974284 9298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.969562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:36915 (local address 127.6.231.194:39721)
0619 12:24:29.969687 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.969692 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.970160 (+ 468us) server_negotiation.cc:407] Connection header received
0619 12:24:29.970292 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.970295 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.970350 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.970428 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:29.970699 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.971372 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.972895 (+ 1523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.973542 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.973668 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.973720 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.974077 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.974163 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:24:29.974284 9297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.969480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:51233 (local address 127.6.231.193:41685)
0619 12:24:29.969598 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:29.969603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:29.969696 (+ 93us) server_negotiation.cc:407] Connection header received
0619 12:24:29.969852 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:29.969855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:29.969902 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:29.969988 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:29.970357 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.971151 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.972895 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.973559 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.973669 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.973720 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:29.974077 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 12:24:29.974155 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:24:29.974310 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.969367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:39721 (local address 127.6.231.195:36915)
0619 12:24:29.970056 (+ 689us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.970070 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.970179 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.970446 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.970449 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.970465 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:29.970576 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.970586 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.971489 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.971492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.972764 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.972770 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.973710 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.973713 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.973891 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.973968 (+ 77us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.974049 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.974147 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"mutex_wait_us":455,"thread_start_us":133,"threads_started":1}
W20260619 12:24:29.974340 9139 tablet.cc:2369] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:24:29.974906 9171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:29.969367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:41685 (local address 127.6.231.195:51233)
0619 12:24:29.969590 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:29.969607 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:29.969741 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:29.970104 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:29.970107 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:29.970124 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:29.970243 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.970249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.971326 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.971328 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.972746 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:29.972758 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:29.973682 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:29.973686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:29.973894 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:29.973963 (+ 69us) client_negotiation.cc:769] Sending connection context
0619 12:24:29.974060 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 12:24:29.974784 (+ 724us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":736}
I20260619 12:24:29.974952 9092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f02cdadfe7425da3b852354a81a587" candidate_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3002b4681a3b4337ba94667b62a849be" is_pre_election: true
I20260619 12:24:29.975104 9092 raft_consensus.cc:2454] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ade6c8d99ca4b6aa9ce40101ba2a99c in term 0.
I20260619 12:24:29.975548 9165 leader_election.cc:302] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3002b4681a3b4337ba94667b62a849be, 4ade6c8d99ca4b6aa9ce40101ba2a99c; no voters:
I20260619 12:24:29.975620 8952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f02cdadfe7425da3b852354a81a587" candidate_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6de249802464d9481b9af89b88ce390" is_pre_election: true
I20260619 12:24:29.975843 8952 raft_consensus.cc:2454] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ade6c8d99ca4b6aa9ce40101ba2a99c in term 0.
I20260619 12:24:29.976063 9308 raft_consensus.cc:2781] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:29.976150 9308 raft_consensus.cc:482] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:29.976197 9308 raft_consensus.cc:3037] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:29.977278 9308 raft_consensus.cc:504] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.977654 9308 leader_election.cc:288] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [CANDIDATE]: Term 1 election: Requested vote from peers f6de249802464d9481b9af89b88ce390 (127.6.231.193:41685), 3002b4681a3b4337ba94667b62a849be (127.6.231.194:39721)
I20260619 12:24:29.978015 9092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f02cdadfe7425da3b852354a81a587" candidate_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3002b4681a3b4337ba94667b62a849be"
I20260619 12:24:29.978034 8952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "32f02cdadfe7425da3b852354a81a587" candidate_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6de249802464d9481b9af89b88ce390"
I20260619 12:24:29.978137 9092 raft_consensus.cc:3037] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:29.978171 8952 raft_consensus.cc:3037] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:29.979277 9092 raft_consensus.cc:2454] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ade6c8d99ca4b6aa9ce40101ba2a99c in term 1.
I20260619 12:24:29.979337 8952 raft_consensus.cc:2454] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ade6c8d99ca4b6aa9ce40101ba2a99c in term 1.
I20260619 12:24:29.979571 9165 leader_election.cc:302] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3002b4681a3b4337ba94667b62a849be, 4ade6c8d99ca4b6aa9ce40101ba2a99c; no voters:
I20260619 12:24:29.979729 9308 raft_consensus.cc:2781] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:29.979943 9308 raft_consensus.cc:686] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 LEADER]: Becoming Leader. State: Replica: 4ade6c8d99ca4b6aa9ce40101ba2a99c, State: Running, Role: LEADER
I20260619 12:24:29.980330 9308 consensus_queue.cc:237] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:29.982822 9324 tablet_replica.cc:439] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: TxnStatusTablet state changed. Reason: New leader 4ade6c8d99ca4b6aa9ce40101ba2a99c. Latest consensus state: current_term: 1 leader_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } } }
I20260619 12:24:29.982993 9324 tablet_replica.cc:442] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:24:29.983440 9326 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 12:24:29.983596 9326 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 12:24:29.983830 8793 catalog_manager.cc:5495] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ade6c8d99ca4b6aa9ce40101ba2a99c (127.6.231.195). New cstate: current_term: 1 leader_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } health_report { overall_health: HEALTHY } } }
I20260619 12:24:30.014194 9300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.010279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37570 (local address 127.6.231.195:39673)
0619 12:24:30.010411 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:30.010415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:30.010522 (+ 107us) server_negotiation.cc:407] Connection header received
0619 12:24:30.010745 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:30.010748 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:30.010834 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:30.010948 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:30.011192 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.011698 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.012563 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.012795 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.012903 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.013235 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:30.013767 (+ 532us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:30.013893 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:30.013963 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 12:24:30.014016 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:24:30.014180 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.010137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:39673 (local address 127.0.0.1:37570)
0619 12:24:30.010442 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:30.010462 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:30.010595 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:30.010960 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:30.010963 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:30.010970 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:24:30.011078 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.011090 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.011811 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.011815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.012442 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.012451 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.012910 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.012914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.013018 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.013065 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:24:30.013867 (+ 802us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:24:30.013884 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 12:24:30.013954 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 12:24:30.014049 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203}
W20260619 12:24:30.015035 8999 tablet.cc:2369] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:24:30.019615 8952 raft_consensus.cc:1261] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Refusing update from remote peer 4ade6c8d99ca4b6aa9ce40101ba2a99c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:30.019615 9092 raft_consensus.cc:1261] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Refusing update from remote peer 4ade6c8d99ca4b6aa9ce40101ba2a99c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:30.020270 9324 consensus_queue.cc:1035] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:30.020522 9308 consensus_queue.cc:1035] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:30.023929 9307 tablet_replica.cc:439] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390: TxnStatusTablet state changed. Reason: New leader 4ade6c8d99ca4b6aa9ce40101ba2a99c. Latest consensus state: current_term: 1 leader_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } } }
I20260619 12:24:30.024075 9307 tablet_replica.cc:442] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:30.025851 9309 tablet_replica.cc:439] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be: TxnStatusTablet state changed. Reason: New leader 4ade6c8d99ca4b6aa9ce40101ba2a99c. Latest consensus state: current_term: 1 leader_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } } }
I20260619 12:24:30.025974 9309 tablet_replica.cc:442] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:30.027127 9324 tablet_replica.cc:439] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } } }
I20260619 12:24:30.027268 9324 tablet_replica.cc:442] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:24:30.027442 9329 mvcc.cc:205] Tried to move back new op lower bound from 7298547179593019392 to 7298547179442106368. Current Snapshot: MvccSnapshot[applied={T|T < 7298547179593019392}]
I20260619 12:24:30.030076 9307 tablet_replica.cc:439] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } } }
I20260619 12:24:30.030210 9307 tablet_replica.cc:442] T 32f02cdadfe7425da3b852354a81a587 P f6de249802464d9481b9af89b88ce390: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:30.030390 9331 mvcc.cc:205] Tried to move back new op lower bound from 7298547179593019392 to 7298547179442106368. Current Snapshot: MvccSnapshot[applied={T|T < 7298547179593019392}]
I20260619 12:24:30.030881 9308 tablet_replica.cc:439] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } } }
I20260619 12:24:30.030998 9308 tablet_replica.cc:442] T 32f02cdadfe7425da3b852354a81a587 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:24:30.033769 9309 tablet_replica.cc:439] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } } }
I20260619 12:24:30.033885 9309 tablet_replica.cc:442] T 32f02cdadfe7425da3b852354a81a587 P 3002b4681a3b4337ba94667b62a849be: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:30.039834 9297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.035945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48522 (local address 127.6.231.193:41685)
0619 12:24:30.036052 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:30.036057 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:30.036070 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:24:30.036139 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:30.036141 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:30.036190 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:30.036270 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:30.036571 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.037201 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.038420 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.038637 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.038728 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.039000 (+ 272us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:30.039406 (+ 406us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:30.039490 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:30.039588 (+ 98us) server_negotiation.cc:299] Negotiation successful
0619 12:24:30.039636 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:24:30.052119 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.048258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:39673 (local address 127.6.231.193:35205)
0619 12:24:30.048614 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:30.048647 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:30.048783 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:30.049082 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:30.049085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:30.049093 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:24:30.049219 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.049225 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.050051 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.050059 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.050772 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.050780 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.051279 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.051283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.051390 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.051434 (+ 44us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:24:30.051784 (+ 350us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:24:30.051793 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 12:24:30.051858 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 12:24:30.051942 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":135,"threads_started":1}
I20260619 12:24:30.052095 9300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.048387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:35205 (local address 127.6.231.195:39673)
0619 12:24:30.048553 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:30.048559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:30.048723 (+ 164us) server_negotiation.cc:407] Connection header received
0619 12:24:30.048930 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:30.048933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:30.048981 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:30.049059 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:30.049330 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.049942 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.050906 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.051158 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.051278 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.051601 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:30.051688 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:30.051768 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:30.051879 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 12:24:30.051934 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260619 12:24:30.059788 9092 raft_consensus.cc:1261] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Refusing update from remote peer f6de249802464d9481b9af89b88ce390: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:30.059989 9216 raft_consensus.cc:1261] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Refusing update from remote peer f6de249802464d9481b9af89b88ce390: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:30.060659 9307 consensus_queue.cc:1035] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:30.060956 9312 consensus_queue.cc:1035] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:30.085275 9297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.081055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:37367 (local address 127.6.231.193:41685)
0619 12:24:30.081170 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:30.081175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:30.081265 (+ 90us) server_negotiation.cc:407] Connection header received
0619 12:24:30.081539 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:30.081542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:30.081610 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:30.081672 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:30.082090 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.082855 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.084059 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.084434 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.084580 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.084848 (+ 268us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:30.084922 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:30.084999 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:30.085101 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 12:24:30.085160 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 12:24:30.085322 9284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.081006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:41685 (local address 127.6.231.195:37367)
0619 12:24:30.081169 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:30.081186 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:30.081288 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:30.081703 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:30.081706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:30.081715 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:24:30.081818 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.081823 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.082957 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.082960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.083929 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.083937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.084554 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.084558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.084647 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.084702 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:24:30.085021 (+ 319us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:24:30.085027 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:24:30.085100 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 12:24:30.085182 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260619 12:24:30.114581 8795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:30.114804 8795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:30.117650 8795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591: Sending DeleteTablet for 3 replicas of tablet f9d39c6ef701471d818c82a91a8b4dd8
I20260619 12:24:30.118428 8930 tablet_service.cc:1514] Processing DeleteTablet for tablet f9d39c6ef701471d818c82a91a8b4dd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:30 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:48514
I20260619 12:24:30.119102 9347 tablet_replica.cc:330] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390: stopping tablet replica
I20260619 12:24:30.119045 9072 tablet_service.cc:1514] Processing DeleteTablet for tablet f9d39c6ef701471d818c82a91a8b4dd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:30 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:51612
I20260619 12:24:30.119567 9347 raft_consensus.cc:2229] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:24:30.119695 9211 tablet_service.cc:1514] Processing DeleteTablet for tablet f9d39c6ef701471d818c82a91a8b4dd8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:30 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:37568
I20260619 12:24:30.120080 9347 raft_consensus.cc:2258] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:30.121007 8795 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:24:30.121594 8795 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:30.122048 9347 ts_tablet_manager.cc:1905] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:30.122994 9348 tablet_replica.cc:330] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be: stopping tablet replica
I20260619 12:24:30.123358 9348 raft_consensus.cc:2229] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:30.123590 9348 raft_consensus.cc:2258] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:30.127568 9347 ts_tablet_manager.cc:1918] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:24:30.127861 9349 tablet_replica.cc:330] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: stopping tablet replica
I20260619 12:24:30.127990 9347 log.cc:1196] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/wal/wals/f9d39c6ef701471d818c82a91a8b4dd8
I20260619 12:24:30.128392 9349 raft_consensus.cc:2229] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:30.128392 8930 tablet_service.cc:1467] Processing CreateTablet for tablet e1b8ea887962446aafbfd29b70338b85 (DEFAULT_TABLE table=test-table [id=3f8a969c4eeb458bbe4d790b7ae4b18b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:30.128727 8930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1b8ea887962446aafbfd29b70338b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:30.128757 9072 tablet_service.cc:1467] Processing CreateTablet for tablet e1b8ea887962446aafbfd29b70338b85 (DEFAULT_TABLE table=test-table [id=3f8a969c4eeb458bbe4d790b7ae4b18b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:30.129024 9072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1b8ea887962446aafbfd29b70338b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:30.131012 9303 tablet_bootstrap.cc:492] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390: Bootstrap starting.
I20260619 12:24:30.131963 9303 tablet_bootstrap.cc:654] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:30.133019 9302 tablet_bootstrap.cc:492] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be: Bootstrap starting.
I20260619 12:24:30.133277 9347 ts_tablet_manager.cc:1939] T f9d39c6ef701471d818c82a91a8b4dd8 P f6de249802464d9481b9af89b88ce390: Deleting consensus metadata
I20260619 12:24:30.129673 9211 tablet_service.cc:1467] Processing CreateTablet for tablet e1b8ea887962446aafbfd29b70338b85 (DEFAULT_TABLE table=test-table [id=3f8a969c4eeb458bbe4d790b7ae4b18b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:30.133656 9211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1b8ea887962446aafbfd29b70338b85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:30.134007 9302 tablet_bootstrap.cc:654] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:30.134723 9348 ts_tablet_manager.cc:1905] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:30.136080 9301 tablet_bootstrap.cc:492] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Bootstrap starting.
I20260619 12:24:30.136989 9301 tablet_bootstrap.cc:654] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:30.137967 9349 raft_consensus.cc:2258] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:30.138718 9348 ts_tablet_manager.cc:1918] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:24:30.138825 9348 log.cc:1196] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/f9d39c6ef701471d818c82a91a8b4dd8
I20260619 12:24:30.139190 9348 ts_tablet_manager.cc:1939] T f9d39c6ef701471d818c82a91a8b4dd8 P 3002b4681a3b4337ba94667b62a849be: Deleting consensus metadata
I20260619 12:24:30.140425 9349 ts_tablet_manager.cc:1905] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:30.141194 8782 catalog_manager.cc:4841] TS f6de249802464d9481b9af89b88ce390 (127.6.231.193:41685): tablet f9d39c6ef701471d818c82a91a8b4dd8 (table test-table [id=6a6959fa34ec4d51a75952e81893b88b]) successfully deleted
I20260619 12:24:30.141211 8781 catalog_manager.cc:4841] TS 3002b4681a3b4337ba94667b62a849be (127.6.231.194:39721): tablet f9d39c6ef701471d818c82a91a8b4dd8 (table test-table [id=6a6959fa34ec4d51a75952e81893b88b]) successfully deleted
I20260619 12:24:30.144315 9302 tablet_bootstrap.cc:492] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be: No bootstrap required, opened a new log
I20260619 12:24:30.144449 9302 ts_tablet_manager.cc:1397] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260619 12:24:30.144505 9349 ts_tablet_manager.cc:1918] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:24:30.144603 9349 log.cc:1196] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/f9d39c6ef701471d818c82a91a8b4dd8
I20260619 12:24:30.144915 9349 ts_tablet_manager.cc:1939] T f9d39c6ef701471d818c82a91a8b4dd8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Deleting consensus metadata
I20260619 12:24:30.144920 9302 raft_consensus.cc:348] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:30.145054 9302 raft_consensus.cc:374] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:30.145092 9302 raft_consensus.cc:729] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3002b4681a3b4337ba94667b62a849be, State: Initialized, Role: FOLLOWER
I20260619 12:24:30.145249 9302 consensus_queue.cc:260] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:30.145555 9302 ts_tablet_manager.cc:1428] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260619 12:24:30.146127 8781 catalog_manager.cc:4841] TS 4ade6c8d99ca4b6aa9ce40101ba2a99c (127.6.231.195:39673): tablet f9d39c6ef701471d818c82a91a8b4dd8 (table test-table [id=6a6959fa34ec4d51a75952e81893b88b]) successfully deleted
I20260619 12:24:30.146965 9301 tablet_bootstrap.cc:492] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: No bootstrap required, opened a new log
I20260619 12:24:30.147063 9301 ts_tablet_manager.cc:1397] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260619 12:24:30.147553 9301 raft_consensus.cc:348] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:30.147675 9301 raft_consensus.cc:374] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:30.147708 9301 raft_consensus.cc:729] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ade6c8d99ca4b6aa9ce40101ba2a99c, State: Initialized, Role: FOLLOWER
I20260619 12:24:30.147831 9301 consensus_queue.cc:260] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:30.148092 9301 ts_tablet_manager.cc:1428] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:30.150427 9303 tablet_bootstrap.cc:492] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390: No bootstrap required, opened a new log
I20260619 12:24:30.150530 9303 ts_tablet_manager.cc:1397] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390: Time spent bootstrapping tablet: real 0.020s user 0.002s sys 0.000s
I20260619 12:24:30.150975 9303 raft_consensus.cc:348] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:30.151093 9303 raft_consensus.cc:374] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:30.151131 9303 raft_consensus.cc:729] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6de249802464d9481b9af89b88ce390, State: Initialized, Role: FOLLOWER
I20260619 12:24:30.151265 9303 consensus_queue.cc:260] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:30.151543 9303 ts_tablet_manager.cc:1428] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:24:30.288012 9309 raft_consensus.cc:482] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:30.288151 9309 raft_consensus.cc:504] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:30.289359 9309 leader_election.cc:288] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6de249802464d9481b9af89b88ce390 (127.6.231.193:41685), 4ade6c8d99ca4b6aa9ce40101ba2a99c (127.6.231.195:39673)
I20260619 12:24:30.293483 9300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.289763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:49751 (local address 127.6.231.195:39673)
0619 12:24:30.289915 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:30.289922 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:30.290209 (+ 287us) server_negotiation.cc:407] Connection header received
0619 12:24:30.290324 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:30.290327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:30.290376 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:30.290461 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:30.290774 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.291244 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.292414 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.292833 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.292958 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.293024 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:30.293243 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 12:24:30.293322 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 12:24:30.293502 9353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.289589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:41685 (local address 127.6.231.194:44915)
0619 12:24:30.289866 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:30.289881 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:30.290002 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:30.290310 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:30.290312 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:30.290330 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:30.290472 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.290478 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.291327 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.291334 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.292314 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.292323 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.293013 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.293017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.293117 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.293134 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 12:24:30.293231 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 12:24:30.293306 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":72,"thread_start_us":90,"threads_started":1}
I20260619 12:24:30.293488 9297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.289686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:44915 (local address 127.6.231.193:41685)
0619 12:24:30.289938 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:30.289944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:30.289961 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:24:30.290127 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:30.290134 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:30.290195 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:30.290277 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:30.290615 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.291209 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.292460 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.292849 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.292981 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.293034 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:30.293262 (+ 228us) server_negotiation.cc:299] Negotiation successful
0619 12:24:30.293340 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153}
I20260619 12:24:30.293509 9354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.289662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:39673 (local address 127.6.231.194:49751)
0619 12:24:30.290025 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:30.290036 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:30.290215 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:30.290489 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:30.290492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:30.290507 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:30.290659 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.290665 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.291376 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.291378 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.292285 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.292293 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.292954 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.292958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.293057 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.293081 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 12:24:30.293217 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 12:24:30.293299 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":3,"thread_start_us":96,"threads_started":1}
I20260619 12:24:30.293977 9216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1b8ea887962446aafbfd29b70338b85" candidate_uuid: "3002b4681a3b4337ba94667b62a849be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" is_pre_election: true
I20260619 12:24:30.293977 8952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1b8ea887962446aafbfd29b70338b85" candidate_uuid: "3002b4681a3b4337ba94667b62a849be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6de249802464d9481b9af89b88ce390" is_pre_election: true
I20260619 12:24:30.294132 9216 raft_consensus.cc:2454] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3002b4681a3b4337ba94667b62a849be in term 0.
I20260619 12:24:30.294132 8952 raft_consensus.cc:2454] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3002b4681a3b4337ba94667b62a849be in term 0.
I20260619 12:24:30.294480 9025 leader_election.cc:302] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3002b4681a3b4337ba94667b62a849be, 4ade6c8d99ca4b6aa9ce40101ba2a99c; no voters:
I20260619 12:24:30.294731 9309 raft_consensus.cc:2781] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:30.294791 9309 raft_consensus.cc:482] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:30.294835 9309 raft_consensus.cc:3037] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:30.295806 9309 raft_consensus.cc:504] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:30.296166 9309 leader_election.cc:288] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [CANDIDATE]: Term 1 election: Requested vote from peers f6de249802464d9481b9af89b88ce390 (127.6.231.193:41685), 4ade6c8d99ca4b6aa9ce40101ba2a99c (127.6.231.195:39673)
I20260619 12:24:30.296512 8952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1b8ea887962446aafbfd29b70338b85" candidate_uuid: "3002b4681a3b4337ba94667b62a849be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6de249802464d9481b9af89b88ce390"
I20260619 12:24:30.296612 9216 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1b8ea887962446aafbfd29b70338b85" candidate_uuid: "3002b4681a3b4337ba94667b62a849be" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c"
I20260619 12:24:30.296696 8952 raft_consensus.cc:3037] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:30.296793 9216 raft_consensus.cc:3037] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:30.297767 8952 raft_consensus.cc:2454] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3002b4681a3b4337ba94667b62a849be in term 1.
I20260619 12:24:30.297813 9216 raft_consensus.cc:2454] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3002b4681a3b4337ba94667b62a849be in term 1.
I20260619 12:24:30.298061 9026 leader_election.cc:302] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3002b4681a3b4337ba94667b62a849be, f6de249802464d9481b9af89b88ce390; no voters:
I20260619 12:24:30.298259 9309 raft_consensus.cc:2781] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:30.298475 9309 raft_consensus.cc:686] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [term 1 LEADER]: Becoming Leader. State: Replica: 3002b4681a3b4337ba94667b62a849be, State: Running, Role: LEADER
I20260619 12:24:30.298756 9309 consensus_queue.cc:237] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:30.301213 8795 catalog_manager.cc:5495] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be reported cstate change: term changed from 0 to 1, leader changed from <none> to 3002b4681a3b4337ba94667b62a849be (127.6.231.194). New cstate: current_term: 1 leader_uuid: "3002b4681a3b4337ba94667b62a849be" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } health_report { overall_health: UNKNOWN } } }
I20260619 12:24:30.333063 9298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.329554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51626 (local address 127.6.231.194:39721)
0619 12:24:30.329689 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:30.329692 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:30.329739 (+ 47us) server_negotiation.cc:407] Connection header received
0619 12:24:30.329886 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:30.329889 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:30.329940 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:30.330030 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:30.330187 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.330685 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.331588 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.331846 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.331937 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.332156 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:30.332647 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:30.332763 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:30.332824 (+ 61us) server_negotiation.cc:299] Negotiation successful
0619 12:24:30.332874 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 12:24:30.344383 9300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.340523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:56663 (local address 127.6.231.195:39673)
0619 12:24:30.340663 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:30.340668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:30.340806 (+ 138us) server_negotiation.cc:407] Connection header received
0619 12:24:30.340986 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:30.340989 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:30.341040 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:30.341288 (+ 248us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:30.341622 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.342198 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.343223 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.343455 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.343569 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.343884 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:30.343959 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:30.344040 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:30.344172 (+ 132us) server_negotiation.cc:299] Negotiation successful
0619 12:24:30.344227 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:24:30.344565 9360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.340360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:39673 (local address 127.6.231.194:56663)
0619 12:24:30.340716 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:30.340730 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:30.340857 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:30.341290 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:30.341293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:30.341304 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:24:30.341494 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.341501 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.342311 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.342318 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.343049 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.343059 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.343570 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.343573 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.343691 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.343736 (+ 45us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:24:30.344065 (+ 329us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:24:30.344072 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:24:30.344144 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 12:24:30.344234 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":133,"threads_started":1}
I20260619 12:24:30.350540 9216 raft_consensus.cc:1261] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Refusing update from remote peer 3002b4681a3b4337ba94667b62a849be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:30.350631 8952 raft_consensus.cc:1261] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Refusing update from remote peer 3002b4681a3b4337ba94667b62a849be: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:30.351114 9355 consensus_queue.cc:1035] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:30.351385 9309 consensus_queue.cc:1035] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:30.370493 9284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.367036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:39721 (local address 127.6.231.195:60733)
0619 12:24:30.367160 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:30.367177 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:30.367255 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:30.367692 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:30.367695 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:30.367706 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:24:30.367812 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.367816 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.368538 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.368541 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.369121 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:30.369128 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.369759 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.369767 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:30.369859 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.369892 (+ 33us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:24:30.370205 (+ 313us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:24:30.370212 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:24:30.370273 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 12:24:30.370353 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260619 12:24:30.370517 9298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:30.367081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:60733 (local address 127.6.231.194:39721)
0619 12:24:30.367402 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:30.367408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:30.367422 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:24:30.367461 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:30.367464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:30.367511 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:30.367579 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:30.367914 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.368429 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.369253 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:30.369609 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:30.369735 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:30.370021 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:30.370104 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:30.370189 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:30.370304 (+ 115us) server_negotiation.cc:299] Negotiation successful
0619 12:24:30.370361 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250}
W20260619 12:24:30.371243 9330 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 12:24:31.927881 8794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:31.928071 8794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:31.930385 8794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591: Sending DeleteTablet for 3 replicas of tablet e1b8ea887962446aafbfd29b70338b85
I20260619 12:24:31.930936 8930 tablet_service.cc:1514] Processing DeleteTablet for tablet e1b8ea887962446aafbfd29b70338b85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:31 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:48514
I20260619 12:24:31.931073 9211 tablet_service.cc:1514] Processing DeleteTablet for tablet e1b8ea887962446aafbfd29b70338b85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:31 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:37568
I20260619 12:24:31.931154 9072 tablet_service.cc:1514] Processing DeleteTablet for tablet e1b8ea887962446aafbfd29b70338b85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:31 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:51612
I20260619 12:24:31.931382 9389 tablet_replica.cc:330] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: stopping tablet replica
I20260619 12:24:31.931550 9389 raft_consensus.cc:2229] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:31.931720 9389 raft_consensus.cc:2258] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:31.931450 9388 tablet_replica.cc:330] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390: stopping tablet replica
I20260619 12:24:31.931972 9388 raft_consensus.cc:2229] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:31.932106 9388 raft_consensus.cc:2258] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:31.932821 9388 ts_tablet_manager.cc:1905] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:31.933323 9390 tablet_replica.cc:330] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be: stopping tablet replica
I20260619 12:24:31.933365 9389 ts_tablet_manager.cc:1905] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:31.933535 9390 raft_consensus.cc:2229] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:24:31.933873 9390 raft_consensus.cc:2258] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:31.935536 9390 ts_tablet_manager.cc:1905] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:31.935843 9388 ts_tablet_manager.cc:1918] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 12:24:31.935843 9389 ts_tablet_manager.cc:1918] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 12:24:31.935920 9389 log.cc:1196] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/e1b8ea887962446aafbfd29b70338b85
I20260619 12:24:31.935982 9388 log.cc:1196] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/wal/wals/e1b8ea887962446aafbfd29b70338b85
I20260619 12:24:31.936270 9389 ts_tablet_manager.cc:1939] T e1b8ea887962446aafbfd29b70338b85 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Deleting consensus metadata
I20260619 12:24:31.936354 9388 ts_tablet_manager.cc:1939] T e1b8ea887962446aafbfd29b70338b85 P f6de249802464d9481b9af89b88ce390: Deleting consensus metadata
I20260619 12:24:31.936987 8782 catalog_manager.cc:4841] TS f6de249802464d9481b9af89b88ce390 (127.6.231.193:41685): tablet e1b8ea887962446aafbfd29b70338b85 (table test-table [id=3f8a969c4eeb458bbe4d790b7ae4b18b]) successfully deleted
I20260619 12:24:31.937031 8781 catalog_manager.cc:4841] TS 4ade6c8d99ca4b6aa9ce40101ba2a99c (127.6.231.195:39673): tablet e1b8ea887962446aafbfd29b70338b85 (table test-table [id=3f8a969c4eeb458bbe4d790b7ae4b18b]) successfully deleted
I20260619 12:24:31.938793 9390 ts_tablet_manager.cc:1918] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260619 12:24:31.938858 9390 log.cc:1196] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/e1b8ea887962446aafbfd29b70338b85
I20260619 12:24:31.939129 9390 ts_tablet_manager.cc:1939] T e1b8ea887962446aafbfd29b70338b85 P 3002b4681a3b4337ba94667b62a849be: Deleting consensus metadata
I20260619 12:24:31.939769 8781 catalog_manager.cc:4841] TS 3002b4681a3b4337ba94667b62a849be (127.6.231.194:39721): tablet e1b8ea887962446aafbfd29b70338b85 (table test-table [id=3f8a969c4eeb458bbe4d790b7ae4b18b]) successfully deleted
Jun 19 12:24:31 dist-test-slave-kr9h krb5kdc[8750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871871, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:24:31 dist-test-slave-kr9h krb5kdc[8750](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871871, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:31.959105 9398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:31.950456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56792 (local address 127.6.231.254:39203)
0619 12:24:31.950759 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:31.950771 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:31.950787 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:24:31.950953 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:31.950955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:31.950999 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:31.951124 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:31.951805 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:31.952355 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:31.953156 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:31.953386 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:31.953543 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:31.955941 (+ 2398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:31.955945 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:31.955947 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:31.955971 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:31.957423 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:31.957864 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:31.957868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:31.957870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:31.957953 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:31.958218 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:31.958221 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:31.958223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:31.958412 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:31.958550 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:31.958783 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 12:24:31.958909 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":120,"threads_started":1}
I20260619 12:24:31.961699 8794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:24:31.962196 8794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:31.966756 8930 tablet_service.cc:1467] Processing CreateTablet for tablet 4a5d965d155449bb9ad39125e7580027 (DEFAULT_TABLE table=test-table [id=1b5cd6031329411a99a8f12d3c140591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:31.967053 8930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a5d965d155449bb9ad39125e7580027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:31.967010 9072 tablet_service.cc:1467] Processing CreateTablet for tablet 4a5d965d155449bb9ad39125e7580027 (DEFAULT_TABLE table=test-table [id=1b5cd6031329411a99a8f12d3c140591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:31.967041 9211 tablet_service.cc:1467] Processing CreateTablet for tablet 4a5d965d155449bb9ad39125e7580027 (DEFAULT_TABLE table=test-table [id=1b5cd6031329411a99a8f12d3c140591]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:31.967273 9211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a5d965d155449bb9ad39125e7580027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:31.967274 9072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4a5d965d155449bb9ad39125e7580027. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:31.970063 9401 tablet_bootstrap.cc:492] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be: Bootstrap starting.
I20260619 12:24:31.970300 9399 tablet_bootstrap.cc:492] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390: Bootstrap starting.
I20260619 12:24:31.970889 9401 tablet_bootstrap.cc:654] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:31.971091 9399 tablet_bootstrap.cc:654] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:31.971500 9400 tablet_bootstrap.cc:492] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Bootstrap starting.
I20260619 12:24:31.972230 9401 tablet_bootstrap.cc:492] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be: No bootstrap required, opened a new log
I20260619 12:24:31.972241 9399 tablet_bootstrap.cc:492] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390: No bootstrap required, opened a new log
I20260619 12:24:31.972324 9399 ts_tablet_manager.cc:1397] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:31.972388 9400 tablet_bootstrap.cc:654] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:31.972393 9401 ts_tablet_manager.cc:1397] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:24:31.972800 9399 raft_consensus.cc:348] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:31.972882 9401 raft_consensus.cc:348] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:31.972944 9399 raft_consensus.cc:374] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:31.972987 9401 raft_consensus.cc:374] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:31.973012 9401 raft_consensus.cc:729] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3002b4681a3b4337ba94667b62a849be, State: Initialized, Role: FOLLOWER
I20260619 12:24:31.972985 9399 raft_consensus.cc:729] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6de249802464d9481b9af89b88ce390, State: Initialized, Role: FOLLOWER
I20260619 12:24:31.973140 9401 consensus_queue.cc:260] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:31.973133 9399 consensus_queue.cc:260] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:31.973474 9401 ts_tablet_manager.cc:1428] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:31.973538 9399 ts_tablet_manager.cc:1428] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:31.973588 9400 tablet_bootstrap.cc:492] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: No bootstrap required, opened a new log
I20260619 12:24:31.973659 9400 ts_tablet_manager.cc:1397] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:24:31.974045 9400 raft_consensus.cc:348] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:31.974215 9400 raft_consensus.cc:374] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:31.974269 9400 raft_consensus.cc:729] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ade6c8d99ca4b6aa9ce40101ba2a99c, State: Initialized, Role: FOLLOWER
I20260619 12:24:31.974442 9400 consensus_queue.cc:260] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:31.974730 9400 ts_tablet_manager.cc:1428] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:31.998737 9324 raft_consensus.cc:482] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:31.998868 9324 raft_consensus.cc:504] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:31.999272 9324 leader_election.cc:288] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6de249802464d9481b9af89b88ce390 (127.6.231.193:41685), 3002b4681a3b4337ba94667b62a849be (127.6.231.194:39721)
I20260619 12:24:31.999701 8952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a5d965d155449bb9ad39125e7580027" candidate_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6de249802464d9481b9af89b88ce390" is_pre_election: true
I20260619 12:24:31.999735 9092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a5d965d155449bb9ad39125e7580027" candidate_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3002b4681a3b4337ba94667b62a849be" is_pre_election: true
I20260619 12:24:31.999864 9092 raft_consensus.cc:2454] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ade6c8d99ca4b6aa9ce40101ba2a99c in term 0.
I20260619 12:24:31.999894 8952 raft_consensus.cc:2454] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ade6c8d99ca4b6aa9ce40101ba2a99c in term 0.
I20260619 12:24:32.000196 9165 leader_election.cc:302] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3002b4681a3b4337ba94667b62a849be, 4ade6c8d99ca4b6aa9ce40101ba2a99c; no voters:
I20260619 12:24:32.000401 9324 raft_consensus.cc:2781] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:32.000471 9324 raft_consensus.cc:482] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:32.000501 9324 raft_consensus.cc:3037] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:32.001643 9324 raft_consensus.cc:504] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:32.001948 9324 leader_election.cc:288] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [CANDIDATE]: Term 1 election: Requested vote from peers f6de249802464d9481b9af89b88ce390 (127.6.231.193:41685), 3002b4681a3b4337ba94667b62a849be (127.6.231.194:39721)
I20260619 12:24:32.002290 8952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a5d965d155449bb9ad39125e7580027" candidate_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6de249802464d9481b9af89b88ce390"
I20260619 12:24:32.002326 9092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4a5d965d155449bb9ad39125e7580027" candidate_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3002b4681a3b4337ba94667b62a849be"
I20260619 12:24:32.002437 8952 raft_consensus.cc:3037] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:32.002446 9092 raft_consensus.cc:3037] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:32.003341 8952 raft_consensus.cc:2454] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ade6c8d99ca4b6aa9ce40101ba2a99c in term 1.
I20260619 12:24:32.003407 9092 raft_consensus.cc:2454] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ade6c8d99ca4b6aa9ce40101ba2a99c in term 1.
I20260619 12:24:32.003661 9166 leader_election.cc:302] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ade6c8d99ca4b6aa9ce40101ba2a99c, f6de249802464d9481b9af89b88ce390; no voters:
I20260619 12:24:32.003846 9324 raft_consensus.cc:2781] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:32.003958 9324 raft_consensus.cc:686] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 LEADER]: Becoming Leader. State: Replica: 4ade6c8d99ca4b6aa9ce40101ba2a99c, State: Running, Role: LEADER
I20260619 12:24:32.004105 9324 consensus_queue.cc:237] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:32.005765 8794 catalog_manager.cc:5495] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ade6c8d99ca4b6aa9ce40101ba2a99c (127.6.231.195). New cstate: current_term: 1 leader_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } health_report { overall_health: HEALTHY } } }
I20260619 12:24:32.020174 9408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:32.017231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37580 (local address 127.6.231.195:39673)
0619 12:24:32.017510 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:32.017522 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:32.017534 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:24:32.017576 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:32.017578 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:32.017618 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:32.017689 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:32.017918 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:32.018423 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:32.019307 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:32.019518 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:32.019611 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:32.019815 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:32.019880 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:32.019933 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:32.020011 (+ 78us) server_negotiation.cc:299] Negotiation successful
0619 12:24:32.020053 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":126,"threads_started":1}
I20260619 12:24:32.028664 9408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:32.025731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:46547 (local address 127.6.231.195:39673)
0619 12:24:32.025832 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:32.025836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:32.025903 (+ 67us) server_negotiation.cc:407] Connection header received
0619 12:24:32.025988 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:32.025990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:32.026030 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:32.026092 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:32.026305 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:32.026800 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:32.027643 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:32.027867 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:32.027949 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:32.028194 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:32.028266 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:32.028333 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:32.028429 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 12:24:32.028474 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 12:24:32.028669 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:32.025595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:39673 (local address 127.6.231.195:46547)
0619 12:24:32.025830 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:32.025842 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:32.025928 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:32.026134 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:32.026136 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:32.026142 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:24:32.026226 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:32.026230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:32.026888 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:32.026890 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:32.027541 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:32.027550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:32.027953 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:32.027955 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:32.028015 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:32.028037 (+ 22us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:24:32.028355 (+ 318us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:24:32.028361 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:24:32.028421 (+ 60us) client_negotiation.cc:240] Negotiation successful
0619 12:24:32.028490 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260619 12:24:32.034065 9092 raft_consensus.cc:1261] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Refusing update from remote peer 4ade6c8d99ca4b6aa9ce40101ba2a99c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:32.034096 8952 raft_consensus.cc:1261] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Refusing update from remote peer 4ade6c8d99ca4b6aa9ce40101ba2a99c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:32.034420 9324 consensus_queue.cc:1035] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:32.034561 9375 consensus_queue.cc:1035] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:32.066974 8794 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:32.067193 8794 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:32.069130 8794 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591: Sending DeleteTablet for 3 replicas of tablet 4a5d965d155449bb9ad39125e7580027
I20260619 12:24:32.069659 8930 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a5d965d155449bb9ad39125e7580027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:32 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:48514
I20260619 12:24:32.069752 9072 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a5d965d155449bb9ad39125e7580027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:32 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:51612
I20260619 12:24:32.069836 9388 tablet_replica.cc:330] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390: stopping tablet replica
I20260619 12:24:32.069947 9388 raft_consensus.cc:2229] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:32.070062 9388 raft_consensus.cc:2258] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:32.070247 9390 tablet_replica.cc:330] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be: stopping tablet replica
I20260619 12:24:32.070237 9211 tablet_service.cc:1514] Processing DeleteTablet for tablet 4a5d965d155449bb9ad39125e7580027 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:32 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:37568
I20260619 12:24:32.070353 9390 raft_consensus.cc:2229] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:32.070416 9389 tablet_replica.cc:330] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: stopping tablet replica
I20260619 12:24:32.070448 9390 raft_consensus.cc:2258] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:32.070531 9389 raft_consensus.cc:2229] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:24:32.070626 8794 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56792:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 12:24:32.070815 9389 raft_consensus.cc:2258] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 12:24:32.071033 8794 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:32.071385 9390 ts_tablet_manager.cc:1905] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:32.071525 9389 ts_tablet_manager.cc:1905] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:32.073566 9389 ts_tablet_manager.cc:1918] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:24:32.073639 9389 log.cc:1196] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/4a5d965d155449bb9ad39125e7580027
I20260619 12:24:32.073920 9389 ts_tablet_manager.cc:1939] T 4a5d965d155449bb9ad39125e7580027 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Deleting consensus metadata
I20260619 12:24:32.073954 9390 ts_tablet_manager.cc:1918] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:24:32.074031 9390 log.cc:1196] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/4a5d965d155449bb9ad39125e7580027
I20260619 12:24:32.074299 9390 ts_tablet_manager.cc:1939] T 4a5d965d155449bb9ad39125e7580027 P 3002b4681a3b4337ba94667b62a849be: Deleting consensus metadata
I20260619 12:24:32.074998 8781 catalog_manager.cc:4841] TS 3002b4681a3b4337ba94667b62a849be (127.6.231.194:39721): tablet 4a5d965d155449bb9ad39125e7580027 (table test-table [id=1b5cd6031329411a99a8f12d3c140591]) successfully deleted
I20260619 12:24:32.075519 9388 ts_tablet_manager.cc:1905] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:32.077914 9388 ts_tablet_manager.cc:1918] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:24:32.077978 9388 log.cc:1196] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1781871857195028-7071-0/minicluster-data/ts-0/wal/wals/4a5d965d155449bb9ad39125e7580027
I20260619 12:24:32.078234 9388 ts_tablet_manager.cc:1939] T 4a5d965d155449bb9ad39125e7580027 P f6de249802464d9481b9af89b88ce390: Deleting consensus metadata
I20260619 12:24:32.078318 9072 tablet_service.cc:1467] Processing CreateTablet for tablet c0ef529589ca4222b4a4b787ea3e35d8 (DEFAULT_TABLE table=test-table [id=8a31edf0c1cd436190ac23b3572e5a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:32.078581 9072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0ef529589ca4222b4a4b787ea3e35d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:32.078716 9211 tablet_service.cc:1467] Processing CreateTablet for tablet c0ef529589ca4222b4a4b787ea3e35d8 (DEFAULT_TABLE table=test-table [id=8a31edf0c1cd436190ac23b3572e5a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:32.078936 9211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0ef529589ca4222b4a4b787ea3e35d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:32.079211 8930 tablet_service.cc:1467] Processing CreateTablet for tablet c0ef529589ca4222b4a4b787ea3e35d8 (DEFAULT_TABLE table=test-table [id=8a31edf0c1cd436190ac23b3572e5a52]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:32.079418 8930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c0ef529589ca4222b4a4b787ea3e35d8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:32.080817 9401 tablet_bootstrap.cc:492] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be: Bootstrap starting.
I20260619 12:24:32.081285 9400 tablet_bootstrap.cc:492] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Bootstrap starting.
I20260619 12:24:32.081390 9399 tablet_bootstrap.cc:492] T c0ef529589ca4222b4a4b787ea3e35d8 P f6de249802464d9481b9af89b88ce390: Bootstrap starting.
I20260619 12:24:32.081693 9401 tablet_bootstrap.cc:654] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:32.082108 9400 tablet_bootstrap.cc:654] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:32.082182 9399 tablet_bootstrap.cc:654] T c0ef529589ca4222b4a4b787ea3e35d8 P f6de249802464d9481b9af89b88ce390: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:32.082652 8782 catalog_manager.cc:4841] TS f6de249802464d9481b9af89b88ce390 (127.6.231.193:41685): tablet 4a5d965d155449bb9ad39125e7580027 (table test-table [id=1b5cd6031329411a99a8f12d3c140591]) successfully deleted
I20260619 12:24:32.083173 9401 tablet_bootstrap.cc:492] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be: No bootstrap required, opened a new log
I20260619 12:24:32.083261 9401 ts_tablet_manager.cc:1397] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260619 12:24:32.083570 8781 catalog_manager.cc:4841] TS 4ade6c8d99ca4b6aa9ce40101ba2a99c (127.6.231.195:39673): tablet 4a5d965d155449bb9ad39125e7580027 (table test-table [id=1b5cd6031329411a99a8f12d3c140591]) successfully deleted
I20260619 12:24:32.083666 9401 raft_consensus.cc:348] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:32.083784 9401 raft_consensus.cc:374] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:32.083819 9401 raft_consensus.cc:729] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3002b4681a3b4337ba94667b62a849be, State: Initialized, Role: FOLLOWER
I20260619 12:24:32.083966 9401 consensus_queue.cc:260] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:32.084199 9399 tablet_bootstrap.cc:492] T c0ef529589ca4222b4a4b787ea3e35d8 P f6de249802464d9481b9af89b88ce390: No bootstrap required, opened a new log
I20260619 12:24:32.084244 9401 ts_tablet_manager.cc:1428] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:32.084268 9399 ts_tablet_manager.cc:1397] T c0ef529589ca4222b4a4b787ea3e35d8 P f6de249802464d9481b9af89b88ce390: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260619 12:24:32.084278 9400 tablet_bootstrap.cc:492] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: No bootstrap required, opened a new log
I20260619 12:24:32.084347 9400 ts_tablet_manager.cc:1397] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 12:24:32.084610 9399 raft_consensus.cc:348] T c0ef529589ca4222b4a4b787ea3e35d8 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:32.084707 9399 raft_consensus.cc:374] T c0ef529589ca4222b4a4b787ea3e35d8 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:32.084736 9399 raft_consensus.cc:729] T c0ef529589ca4222b4a4b787ea3e35d8 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6de249802464d9481b9af89b88ce390, State: Initialized, Role: FOLLOWER
I20260619 12:24:32.084707 9400 raft_consensus.cc:348] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:32.084833 9400 raft_consensus.cc:374] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:32.084842 9399 consensus_queue.cc:260] T c0ef529589ca4222b4a4b787ea3e35d8 P f6de249802464d9481b9af89b88ce390 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:32.084874 9400 raft_consensus.cc:729] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ade6c8d99ca4b6aa9ce40101ba2a99c, State: Initialized, Role: FOLLOWER
I20260619 12:24:32.084990 9400 consensus_queue.cc:260] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:32.085074 9399 ts_tablet_manager.cc:1428] T c0ef529589ca4222b4a4b787ea3e35d8 P f6de249802464d9481b9af89b88ce390: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 12:24:32.085238 9400 ts_tablet_manager.cc:1428] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:32.188763 9375 raft_consensus.cc:482] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:32.188925 9375 raft_consensus.cc:504] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:32.189507 9375 leader_election.cc:288] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f6de249802464d9481b9af89b88ce390 (127.6.231.193:41685), 3002b4681a3b4337ba94667b62a849be (127.6.231.194:39721)
I20260619 12:24:32.189949 8952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0ef529589ca4222b4a4b787ea3e35d8" candidate_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6de249802464d9481b9af89b88ce390" is_pre_election: true
I20260619 12:24:32.189978 9091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0ef529589ca4222b4a4b787ea3e35d8" candidate_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3002b4681a3b4337ba94667b62a849be" is_pre_election: true
I20260619 12:24:32.190109 8952 raft_consensus.cc:2454] T c0ef529589ca4222b4a4b787ea3e35d8 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ade6c8d99ca4b6aa9ce40101ba2a99c in term 0.
I20260619 12:24:32.190150 9091 raft_consensus.cc:2454] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4ade6c8d99ca4b6aa9ce40101ba2a99c in term 0.
I20260619 12:24:32.190438 9165 leader_election.cc:302] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3002b4681a3b4337ba94667b62a849be, 4ade6c8d99ca4b6aa9ce40101ba2a99c; no voters:
I20260619 12:24:32.190630 9375 raft_consensus.cc:2781] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:32.190711 9375 raft_consensus.cc:482] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:32.190766 9375 raft_consensus.cc:3037] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:32.191781 9375 raft_consensus.cc:504] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:32.192157 9375 leader_election.cc:288] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [CANDIDATE]: Term 1 election: Requested vote from peers f6de249802464d9481b9af89b88ce390 (127.6.231.193:41685), 3002b4681a3b4337ba94667b62a849be (127.6.231.194:39721)
I20260619 12:24:32.192458 8952 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0ef529589ca4222b4a4b787ea3e35d8" candidate_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6de249802464d9481b9af89b88ce390"
I20260619 12:24:32.192577 8952 raft_consensus.cc:3037] T c0ef529589ca4222b4a4b787ea3e35d8 P f6de249802464d9481b9af89b88ce390 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:32.192556 9091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c0ef529589ca4222b4a4b787ea3e35d8" candidate_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3002b4681a3b4337ba94667b62a849be"
I20260619 12:24:32.192699 9091 raft_consensus.cc:3037] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:32.193806 8952 raft_consensus.cc:2454] T c0ef529589ca4222b4a4b787ea3e35d8 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ade6c8d99ca4b6aa9ce40101ba2a99c in term 1.
I20260619 12:24:32.193949 9091 raft_consensus.cc:2454] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ade6c8d99ca4b6aa9ce40101ba2a99c in term 1.
I20260619 12:24:32.194173 9166 leader_election.cc:302] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ade6c8d99ca4b6aa9ce40101ba2a99c, f6de249802464d9481b9af89b88ce390; no voters:
I20260619 12:24:32.194370 9375 raft_consensus.cc:2781] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:32.194455 9375 raft_consensus.cc:686] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 LEADER]: Becoming Leader. State: Replica: 4ade6c8d99ca4b6aa9ce40101ba2a99c, State: Running, Role: LEADER
I20260619 12:24:32.194590 9375 consensus_queue.cc:237] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } }
I20260619 12:24:32.196281 8795 catalog_manager.cc:5495] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c reported cstate change: term changed from 0 to 1, leader changed from <none> to 4ade6c8d99ca4b6aa9ce40101ba2a99c (127.6.231.195). New cstate: current_term: 1 leader_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ade6c8d99ca4b6aa9ce40101ba2a99c" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39673 } health_report { overall_health: HEALTHY } } }
I20260619 12:24:32.217653 9091 raft_consensus.cc:1261] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Refusing update from remote peer 4ade6c8d99ca4b6aa9ce40101ba2a99c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:32.217921 8952 raft_consensus.cc:1261] T c0ef529589ca4222b4a4b787ea3e35d8 P f6de249802464d9481b9af89b88ce390 [term 1 FOLLOWER]: Refusing update from remote peer 4ade6c8d99ca4b6aa9ce40101ba2a99c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:32.218041 9369 consensus_queue.cc:1035] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [LEADER]: Connected to new peer: Peer: permanent_uuid: "3002b4681a3b4337ba94667b62a849be" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 39721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:32.218355 9369 consensus_queue.cc:1035] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6de249802464d9481b9af89b88ce390" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 41685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 12:24:32.230319 9367 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260619 12:24:33.777156 8795 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:33.777352 8795 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56792:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:33.779762 8795 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d9b8e19186fc463194cf3d594fb03591: Sending DeleteTablet for 3 replicas of tablet c0ef529589ca4222b4a4b787ea3e35d8
I20260619 12:24:33.780371 8930 tablet_service.cc:1514] Processing DeleteTablet for tablet c0ef529589ca4222b4a4b787ea3e35d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:33 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:48514
I20260619 12:24:33.780483 9072 tablet_service.cc:1514] Processing DeleteTablet for tablet c0ef529589ca4222b4a4b787ea3e35d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:33 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:51612
I20260619 12:24:33.780791 9211 tablet_service.cc:1514] Processing DeleteTablet for tablet c0ef529589ca4222b4a4b787ea3e35d8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:33 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:37568
I20260619 12:24:33.780846 9442 tablet_replica.cc:330] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be: stopping tablet replica
I20260619 12:24:33.780894 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 8867
I20260619 12:24:33.780994 9442 raft_consensus.cc:2229] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:33.781109 9442 raft_consensus.cc:2258] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:33.781630 9443 tablet_replica.cc:330] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: stopping tablet replica
I20260619 12:24:33.781770 9443 raft_consensus.cc:2229] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:24:33.781936 9442 ts_tablet_manager.cc:1905] T c0ef529589ca4222b4a4b787ea3e35d8 P 3002b4681a3b4337ba94667b62a849be: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:33.781976 9443 raft_consensus.cc:2258] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:33.782642 9443 ts_tablet_manager.cc:1905] T c0ef529589ca4222b4a4b787ea3e35d8 P 4ade6c8d99ca4b6aa9ce40101ba2a99c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 12:24:33.789049 8782 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:41685 (error 108)
I20260619 12:24:33.789393 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 9007
I20260619 12:24:33.791682 9445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:33.791235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:41685 (local address 127.0.0.1:48528)
0619 12:24:33.791516 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:33.791589 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:41685: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":85,"threads_started":1}
W20260619 12:24:33.791947 8782 catalog_manager.cc:4584] TS f6de249802464d9481b9af89b88ce390 (127.6.231.193:41685): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c0ef529589ca4222b4a4b787ea3e35d8: Network error: Client connection negotiation failed: client connection to 127.6.231.193:41685: connect: Connection refused (error 111)
I20260619 12:24:33.803967 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 9147
I20260619 12:24:33.805063 9445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:33.804139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:39721 (local address 127.0.0.1:51634)
0619 12:24:33.804858 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:33.804932 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.194:39721: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":601}
I20260619 12:24:33.814113 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 8763
2026-06-19T12:24:33Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8882 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:24:34 dist-test-slave-kr9h krb5kdc[9448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:24:34 dist-test-slave-kr9h krb5kdc[9448](info): set up 2 sockets
Jun 19 12:24:34 dist-test-slave-kr9h krb5kdc[9448](info): commencing operation
krb5kdc: starting...
W20260619 12:24:35.425932 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.537s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:24:35 dist-test-slave-kr9h krb5kdc[9448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:24:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:24:35.554951 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:45117
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:45117
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:35.666179 9461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:35.666517 9461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:35.666586 9461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:35.670713 9461 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:35.670809 9461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:35.670846 9461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:35.670902 9461 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:35.670943 9461 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:35.676468 9461 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:45117
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:45117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:35.677634 9461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:35.678732 9461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:35.687311 9467 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:35.687335 9466 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:35.687332 9469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:35.687549 9461 server_base.cc:1034] running on GCE node
I20260619 12:24:35.688068 9461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:35.689198 9461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:35.690388 9461 hybrid_clock.cc:648] HybridClock initialized: now 1781871875690368 us; error 37 us; skew 500 ppm
Jun 19 12:24:35 dist-test-slave-kr9h krb5kdc[9448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871875, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:35.693075 9461 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:24:35.694074 9461 webserver.cc:460] Webserver started at http://127.6.231.254:42895/ using document root <none> and password file <none>
I20260619 12:24:35.694638 9461 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:35.694725 9461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:35.694950 9461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:35.697021 9461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "cd9f3b2473cb40dc805fbf3220cb5669"
format_stamp: "Formatted at 2026-06-19 12:24:35 on dist-test-slave-kr9h"
server_key: "a51a591d571e2fefbce6c80dd7e11e43"
server_key_iv: "07201555821351b2c1590915114cdfd9"
server_key_version: "encryptionkey@0"
I20260619 12:24:35.697638 9461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "cd9f3b2473cb40dc805fbf3220cb5669"
format_stamp: "Formatted at 2026-06-19 12:24:35 on dist-test-slave-kr9h"
server_key: "a51a591d571e2fefbce6c80dd7e11e43"
server_key_iv: "07201555821351b2c1590915114cdfd9"
server_key_version: "encryptionkey@0"
I20260619 12:24:35.701412 9461 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260619 12:24:35.703948 9476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:35.705361 9461 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 12:24:35.705530 9461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "cd9f3b2473cb40dc805fbf3220cb5669"
format_stamp: "Formatted at 2026-06-19 12:24:35 on dist-test-slave-kr9h"
server_key: "a51a591d571e2fefbce6c80dd7e11e43"
server_key_iv: "07201555821351b2c1590915114cdfd9"
server_key_version: "encryptionkey@0"
I20260619 12:24:35.705658 9461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:35.723757 9461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:35.726728 9461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:35.726931 9461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:35.734107 9461 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:45117
I20260619 12:24:35.734126 9528 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:45117 every 8 connection(s)
I20260619 12:24:35.735086 9461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:35.737808 9529 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:35.741259 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 9461
I20260619 12:24:35.741369 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:35.741673 7071 external_mini_cluster.cc:1428] Setting key 8f3073377d3405c596cce227fdcb3469
I20260619 12:24:35.743764 9529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669: Bootstrap starting.
I20260619 12:24:35.745955 9529 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:35.746660 9529 log.cc:824] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:35.748389 9529 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669: No bootstrap required, opened a new log
Jun 19 12:24:35 dist-test-slave-kr9h krb5kdc[9448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871875, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:35.752142 9529 raft_consensus.cc:348] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd9f3b2473cb40dc805fbf3220cb5669" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45117 } }
I20260619 12:24:35.752386 9529 raft_consensus.cc:374] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:35.752461 9529 raft_consensus.cc:729] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd9f3b2473cb40dc805fbf3220cb5669, State: Initialized, Role: FOLLOWER
I20260619 12:24:35.752871 9529 consensus_queue.cc:260] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd9f3b2473cb40dc805fbf3220cb5669" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45117 } }
I20260619 12:24:35.753012 9529 raft_consensus.cc:388] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:35.753089 9529 raft_consensus.cc:482] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:35.753196 9529 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:35.754083 9529 raft_consensus.cc:504] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd9f3b2473cb40dc805fbf3220cb5669" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45117 } }
I20260619 12:24:35.754431 9529 leader_election.cc:302] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cd9f3b2473cb40dc805fbf3220cb5669; no voters:
I20260619 12:24:35.754788 9529 leader_election.cc:288] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:35.754873 9534 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:35.755030 9534 raft_consensus.cc:686] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [term 1 LEADER]: Becoming Leader. State: Replica: cd9f3b2473cb40dc805fbf3220cb5669, State: Running, Role: LEADER
I20260619 12:24:35.755375 9534 consensus_queue.cc:237] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd9f3b2473cb40dc805fbf3220cb5669" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45117 } }
I20260619 12:24:35.755574 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:35.743133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47970 (local address 127.6.231.254:45117)
0619 12:24:35.743561 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:35.743576 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:35.743612 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:24:35.744284 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:35.744302 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:35.744626 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:35.744965 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:35.745788 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:35.746866 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:35.747704 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:35.748175 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:35.748344 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:35.750803 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:35.750808 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:35.750821 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:35.750847 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:35.752530 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:35.752980 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:35.752985 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:35.753008 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:35.753152 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:35.753451 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:35.753458 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:35.753462 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:35.753858 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:35.754072 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:35.754303 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 12:24:35.754623 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":117,"threads_started":1}
I20260619 12:24:35.756023 9529 sys_catalog.cc:564] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:35.757009 9536 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cd9f3b2473cb40dc805fbf3220cb5669. Latest consensus state: current_term: 1 leader_uuid: "cd9f3b2473cb40dc805fbf3220cb5669" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd9f3b2473cb40dc805fbf3220cb5669" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45117 } } }
I20260619 12:24:35.757130 9536 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:35.757555 9535 sys_catalog.cc:455] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cd9f3b2473cb40dc805fbf3220cb5669" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cd9f3b2473cb40dc805fbf3220cb5669" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45117 } } }
I20260619 12:24:35.757651 9535 sys_catalog.cc:458] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669 [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:35.757701 9543 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:35.760696 9543 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:35.766532 9543 catalog_manager.cc:1347] Generated new cluster ID: 504fb4c268d04cac9914a40090dc7294
I20260619 12:24:35.766604 9543 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:35.784384 9543 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:35.785377 9543 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:35.796394 9543 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cd9f3b2473cb40dc805fbf3220cb5669: Generated new TSK 0
I20260619 12:24:35.797180 9543 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:24:35.827014 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45117
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:35.941926 9555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:35.942214 9555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:35.942265 9555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:35.946831 9555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:35.946909 9555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:35.947003 9555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:24:35.953002 9555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:35.954260 9555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:35.955466 9555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:24:35.966176 9555 server_base.cc:1034] running on GCE node
W20260619 12:24:35.966132 9560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:35.966133 9563 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:35.966132 9561 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:35.966837 9555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:35.967543 9555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:35.968743 9555 hybrid_clock.cc:648] HybridClock initialized: now 1781871875968715 us; error 39 us; skew 500 ppm
Jun 19 12:24:35 dist-test-slave-kr9h krb5kdc[9448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871875, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:35.971704 9555 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:24:35.972592 9555 webserver.cc:460] Webserver started at http://127.6.231.193:34153/ using document root <none> and password file <none>
I20260619 12:24:35.973169 9555 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:35.973228 9555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:35.973479 9555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:35.975490 9555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "0ab19e6cec754cddb0eab92bca8b6bcd"
format_stamp: "Formatted at 2026-06-19 12:24:35 on dist-test-slave-kr9h"
server_key: "a023651bf202612f2f58c0f93801a459"
server_key_iv: "dc99ca833231e6eedd33ba3837f011cf"
server_key_version: "encryptionkey@0"
I20260619 12:24:35.976055 9555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "0ab19e6cec754cddb0eab92bca8b6bcd"
format_stamp: "Formatted at 2026-06-19 12:24:35 on dist-test-slave-kr9h"
server_key: "a023651bf202612f2f58c0f93801a459"
server_key_iv: "dc99ca833231e6eedd33ba3837f011cf"
server_key_version: "encryptionkey@0"
I20260619 12:24:35.979892 9555 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:24:35.982472 9570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:35.983803 9555 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:35.983922 9555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "0ab19e6cec754cddb0eab92bca8b6bcd"
format_stamp: "Formatted at 2026-06-19 12:24:35 on dist-test-slave-kr9h"
server_key: "a023651bf202612f2f58c0f93801a459"
server_key_iv: "dc99ca833231e6eedd33ba3837f011cf"
server_key_version: "encryptionkey@0"
I20260619 12:24:35.984030 9555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:36.007975 9555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:36.011559 9555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:36.011761 9555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:36.012398 9555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:36.013255 9555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:36.013312 9555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:36.013355 9555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:36.013409 9555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:36.022488 9555 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:35507
I20260619 12:24:36.022506 9683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:35507 every 8 connection(s)
I20260619 12:24:36.023694 9555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
Jun 19 12:24:36 dist-test-slave-kr9h krb5kdc[9448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871875, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:36.033984 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 9555
I20260619 12:24:36.034071 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:24:36.034348 7071 external_mini_cluster.cc:1428] Setting key 8a094f31d8284b050572ead3122b8e73
I20260619 12:24:36.038395 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:36.025859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:56693 (local address 127.6.231.254:45117)
0619 12:24:36.026015 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:36.026022 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:36.026904 (+ 882us) server_negotiation.cc:407] Connection header received
0619 12:24:36.028270 (+ 1366us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:36.028274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:36.028335 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:36.028417 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:36.029665 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:36.030217 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:36.031355 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:36.031595 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:36.031702 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:36.034278 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:36.034282 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:36.034285 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:36.034318 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:36.036108 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:36.036627 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:36.036632 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:36.036634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:36.036720 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:36.037072 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:36.037075 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:36.037077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:36.037328 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:36.037455 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:36.038036 (+ 581us) server_negotiation.cc:299] Negotiation successful
0619 12:24:36.038196 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 12:24:36.039201 9686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:36.026322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45117 (local address 127.6.231.193:56693)
0619 12:24:36.026757 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:36.026798 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:36.028046 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:36.028596 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:36.028602 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:36.028974 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:36.029524 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:36.029540 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:36.030342 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:36.030346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:36.031233 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:36.031242 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:36.031743 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:36.031747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:36.031917 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:36.032477 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:36.032493 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:36.034083 (+ 1590us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:36.036224 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:36.036230 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:36.036246 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:36.036516 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:36.036815 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:36.036818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:36.036820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:36.036964 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:36.037513 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:36.037519 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:36.037790 (+ 271us) client_negotiation.cc:769] Sending connection context
0619 12:24:36.037963 (+ 173us) client_negotiation.cc:240] Negotiation successful
0619 12:24:36.038229 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":113,"threads_started":1}
I20260619 12:24:36.040557 9684 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45117
I20260619 12:24:36.040814 9684 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:36.041303 9684 heartbeater.cc:511] Master 127.6.231.254:45117 requested a full tablet report, sending...
I20260619 12:24:36.042856 9493 ts_manager.cc:194] Registered new tserver with Master: 0ab19e6cec754cddb0eab92bca8b6bcd (127.6.231.193:35507)
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
I20260619 12:24:36.044034 9493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:56693
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:24:36.062233 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45117
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:36.177506 9689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:36.177771 9689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:36.177842 9689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:36.181836 9689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:36.181900 9689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:36.181995 9689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:24:36.187443 9689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:36.188547 9689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:36.189592 9689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:36.198818 9695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:36.198944 9689 server_base.cc:1034] running on GCE node
W20260619 12:24:36.198856 9694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:36.198868 9697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:36.199539 9689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:36.200201 9689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:36.201354 9689 hybrid_clock.cc:648] HybridClock initialized: now 1781871876201334 us; error 32 us; skew 500 ppm
Jun 19 12:24:36 dist-test-slave-kr9h krb5kdc[9448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871876, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:36.204110 9689 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:24:36.205073 9689 webserver.cc:460] Webserver started at http://127.6.231.194:35995/ using document root <none> and password file <none>
I20260619 12:24:36.205878 9689 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:36.205977 9689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:36.206183 9689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:36.208225 9689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "2f49bb34a1514e7a9275e2db1967d148"
format_stamp: "Formatted at 2026-06-19 12:24:36 on dist-test-slave-kr9h"
server_key: "3aac09ccbc9f19343c4d0d53f2907ab3"
server_key_iv: "3b338819cd67b096ef4453d2b39d91a4"
server_key_version: "encryptionkey@0"
I20260619 12:24:36.208796 9689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "2f49bb34a1514e7a9275e2db1967d148"
format_stamp: "Formatted at 2026-06-19 12:24:36 on dist-test-slave-kr9h"
server_key: "3aac09ccbc9f19343c4d0d53f2907ab3"
server_key_iv: "3b338819cd67b096ef4453d2b39d91a4"
server_key_version: "encryptionkey@0"
I20260619 12:24:36.212606 9689 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260619 12:24:36.215128 9704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:36.216346 9689 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:36.216471 9689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "2f49bb34a1514e7a9275e2db1967d148"
format_stamp: "Formatted at 2026-06-19 12:24:36 on dist-test-slave-kr9h"
server_key: "3aac09ccbc9f19343c4d0d53f2907ab3"
server_key_iv: "3b338819cd67b096ef4453d2b39d91a4"
server_key_version: "encryptionkey@0"
I20260619 12:24:36.216579 9689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:36.239048 9689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:36.242266 9689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:36.242462 9689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:36.243074 9689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:36.243889 9689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:36.243953 9689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:36.244022 9689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:36.244069 9689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:36.253208 9689 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:41461
I20260619 12:24:36.253242 9817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:41461 every 8 connection(s)
I20260619 12:24:36.254184 9689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:24:36.255390 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 9689
I20260619 12:24:36.255510 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:24:36.255770 7071 external_mini_cluster.cc:1428] Setting key 108623e696b5331e16672779d8ba5099
Jun 19 12:24:36 dist-test-slave-kr9h krb5kdc[9448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871876, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
I20260619 12:24:36.269841 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:36.256416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:33721 (local address 127.6.231.254:45117)
0619 12:24:36.256563 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:36.256568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:36.257897 (+ 1329us) server_negotiation.cc:407] Connection header received
0619 12:24:36.259672 (+ 1775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:36.259675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:36.259732 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:36.259814 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:36.261093 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:36.261775 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:36.262965 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:36.263206 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:36.263316 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:36.266177 (+ 2861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:36.266180 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:36.266182 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:36.266205 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:36.267525 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:36.267985 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:36.267988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:36.267989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:36.268061 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:36.268498 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:36.268501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:36.268503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:36.268728 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:36.269045 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:36.269544 (+ 499us) server_negotiation.cc:299] Negotiation successful
0619 12:24:36.269671 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 12:24:36.270982 9820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:36.256880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45117 (local address 127.6.231.194:33721)
0619 12:24:36.257761 (+ 881us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:36.257804 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:36.259425 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:36.260011 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:36.260024 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:36.260444 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:36.260951 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:36.260965 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:36.261895 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:36.261899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:36.262843 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:36.262852 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:36.263351 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:36.263356 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:36.263534 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:36.264202 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:36.264218 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:36.265994 (+ 1776us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:36.267626 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:36.267631 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:36.267642 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:36.267900 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:36.268188 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:36.268191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:36.268193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:36.268377 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:36.269029 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:36.269035 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:36.269276 (+ 241us) client_negotiation.cc:769] Sending connection context
0619 12:24:36.269505 (+ 229us) client_negotiation.cc:240] Negotiation successful
0619 12:24:36.269864 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":141,"threads_started":1}
I20260619 12:24:36.272290 9818 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45117
I20260619 12:24:36.272542 9818 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:36.273093 9818 heartbeater.cc:511] Master 127.6.231.254:45117 requested a full tablet report, sending...
I20260619 12:24:36.274245 9493 ts_manager.cc:194] Registered new tserver with Master: 2f49bb34a1514e7a9275e2db1967d148 (127.6.231.194:41461)
I20260619 12:24:36.274865 9493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:33721
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:24:36.284451 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45117
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:36.397266 9823 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:36.397531 9823 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:36.397575 9823 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:36.401499 9823 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:36.401551 9823 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:36.401608 9823 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:24:36.406826 9823 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45117
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9823
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:36.407833 9823 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:36.408932 9823 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:36.418438 9831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:36.418426 9829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:36.418424 9828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:36.418658 9823 server_base.cc:1034] running on GCE node
I20260619 12:24:36.419126 9823 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:36.419843 9823 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:36.421023 9823 hybrid_clock.cc:648] HybridClock initialized: now 1781871876421009 us; error 35 us; skew 500 ppm
Jun 19 12:24:36 dist-test-slave-kr9h krb5kdc[9448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871876, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:36.423810 9823 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:24:36.424707 9823 webserver.cc:460] Webserver started at http://127.6.231.195:33389/ using document root <none> and password file <none>
I20260619 12:24:36.425271 9823 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:36.425359 9823 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:36.425603 9823 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:36.427635 9823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "65bedda4ae2342fa8e6665398a34457b"
format_stamp: "Formatted at 2026-06-19 12:24:36 on dist-test-slave-kr9h"
server_key: "fbe8adcb98969f581db7149583b574b0"
server_key_iv: "ea6ac963af82284a55443fdb548997b3"
server_key_version: "encryptionkey@0"
I20260619 12:24:36.428192 9823 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "65bedda4ae2342fa8e6665398a34457b"
format_stamp: "Formatted at 2026-06-19 12:24:36 on dist-test-slave-kr9h"
server_key: "fbe8adcb98969f581db7149583b574b0"
server_key_iv: "ea6ac963af82284a55443fdb548997b3"
server_key_version: "encryptionkey@0"
I20260619 12:24:36.431893 9823 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 12:24:36.434403 9838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:36.435647 9823 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260619 12:24:36.435775 9823 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "65bedda4ae2342fa8e6665398a34457b"
format_stamp: "Formatted at 2026-06-19 12:24:36 on dist-test-slave-kr9h"
server_key: "fbe8adcb98969f581db7149583b574b0"
server_key_iv: "ea6ac963af82284a55443fdb548997b3"
server_key_version: "encryptionkey@0"
I20260619 12:24:36.435881 9823 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:36.453306 9823 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:36.456665 9823 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:36.456866 9823 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:36.457504 9823 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:36.458386 9823 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:36.458459 9823 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:36.458527 9823 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:36.458575 9823 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:36.467643 9823 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:34925
I20260619 12:24:36.467665 9951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:34925 every 8 connection(s)
I20260619 12:24:36.468564 9823 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:24:36.470846 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 9823
I20260619 12:24:36.470961 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:24:36.471297 7071 external_mini_cluster.cc:1428] Setting key d1c287e1b2bcb572379d3ebfa99f5e9a
Jun 19 12:24:36 dist-test-slave-kr9h krb5kdc[9448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871876, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:36.482569 9532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:36.470424 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:42355 (local address 127.6.231.254:45117)
0619 12:24:36.470558 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:36.470561 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:36.471623 (+ 1062us) server_negotiation.cc:407] Connection header received
0619 12:24:36.473409 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:36.473413 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:36.473494 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:36.473572 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:36.474720 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:36.475255 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:36.476390 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:36.476620 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:36.476736 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:36.479156 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:36.479160 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:36.479163 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:36.479193 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:36.480527 (+ 1334us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:36.481024 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:36.481029 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:36.481031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:36.481111 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:36.481459 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:36.481462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:36.481464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:36.481643 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:36.481757 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:36.482267 (+ 510us) server_negotiation.cc:299] Negotiation successful
0619 12:24:36.482389 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:24:36.483389 9954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:36.470933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45117 (local address 127.6.231.195:42355)
0619 12:24:36.471471 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:36.471512 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:36.473204 (+ 1692us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:36.473741 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:36.473748 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:36.474094 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:36.474580 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:36.474592 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:36.475377 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:36.475380 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:36.476269 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:36.476278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:36.476772 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:36.476775 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:36.476952 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:36.477586 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:36.477601 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:36.478994 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:36.480641 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:36.480648 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:36.480664 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:36.480924 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:36.481201 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:36.481204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:36.481206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:36.481342 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:36.481763 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:36.481772 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:36.482022 (+ 250us) client_negotiation.cc:769] Sending connection context
0619 12:24:36.482225 (+ 203us) client_negotiation.cc:240] Negotiation successful
0619 12:24:36.482583 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":150,"threads_started":1}
I20260619 12:24:36.484584 9952 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45117
I20260619 12:24:36.484851 9952 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:36.485373 9952 heartbeater.cc:511] Master 127.6.231.254:45117 requested a full tablet report, sending...
I20260619 12:24:36.486418 9492 ts_manager.cc:194] Registered new tserver with Master: 65bedda4ae2342fa8e6665398a34457b (127.6.231.195:34925)
I20260619 12:24:36.487027 9492 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:42355
I20260619 12:24:36.496927 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 12:24:36.509569 9532 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:24:36.508014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47984 (local address 127.6.231.254:45117)
0619 12:24:36.508191 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:36.508197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:36.508351 (+ 154us) server_negotiation.cc:407] Connection header received
0619 12:24:36.508519 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:36.508526 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:36.508568 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:36.508653 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:36.509410 (+ 757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:47984: BlockingRecv error: recv got EOF from 127.0.0.1:47984 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 12:24:36.510581 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 9555
I20260619 12:24:36.516156 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 9689
I20260619 12:24:36.521469 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 9823
I20260619 12:24:36.526840 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 9461
2026-06-19T12:24:36Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2659 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:24:36 dist-test-slave-kr9h krb5kdc[9964](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:24:36 dist-test-slave-kr9h krb5kdc[9964](info): set up 2 sockets
Jun 19 12:24:36 dist-test-slave-kr9h krb5kdc[9964](info): commencing operation
krb5kdc: starting...
W20260619 12:24:37.569408 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:24:37 dist-test-slave-kr9h krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:24:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:24:37.690510 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:42255
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:46801
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:42255
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:37.800848 9977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:37.801172 9977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:37.801254 9977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:37.805330 9977 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:37.805445 9977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:37.805510 9977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:37.805639 9977 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:37.805696 9977 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:37.811092 9977 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:46801
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:42255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:42255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:37.812203 9977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:37.813251 9977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:37.821630 9983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:37.821707 9982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:37.821762 9977 server_base.cc:1034] running on GCE node
W20260619 12:24:37.821709 9985 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:37.822415 9977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:37.823535 9977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:37.824728 9977 hybrid_clock.cc:648] HybridClock initialized: now 1781871877824714 us; error 28 us; skew 500 ppm
Jun 19 12:24:37 dist-test-slave-kr9h krb5kdc[9964](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871877, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:37.827505 9977 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:24:37.828413 9977 webserver.cc:460] Webserver started at http://127.6.231.254:34191/ using document root <none> and password file <none>
I20260619 12:24:37.828969 9977 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:37.829056 9977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:37.829279 9977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:37.831391 9977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "04fcd9df24054f9bbce9c203ce136dea"
format_stamp: "Formatted at 2026-06-19 12:24:37 on dist-test-slave-kr9h"
server_key: "ac62e2ed1e6b7ae6376a8e04ad92642f"
server_key_iv: "53f0e9f0be3d5512ee8ca43223d36c32"
server_key_version: "encryptionkey@0"
I20260619 12:24:37.831959 9977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "04fcd9df24054f9bbce9c203ce136dea"
format_stamp: "Formatted at 2026-06-19 12:24:37 on dist-test-slave-kr9h"
server_key: "ac62e2ed1e6b7ae6376a8e04ad92642f"
server_key_iv: "53f0e9f0be3d5512ee8ca43223d36c32"
server_key_version: "encryptionkey@0"
I20260619 12:24:37.835628 9977 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 12:24:37.838070 9992 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:37.839337 9977 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:24:37.839473 9977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "04fcd9df24054f9bbce9c203ce136dea"
format_stamp: "Formatted at 2026-06-19 12:24:37 on dist-test-slave-kr9h"
server_key: "ac62e2ed1e6b7ae6376a8e04ad92642f"
server_key_iv: "53f0e9f0be3d5512ee8ca43223d36c32"
server_key_version: "encryptionkey@0"
I20260619 12:24:37.839593 9977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:37.854636 9977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:37.857723 9977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:37.857918 9977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:37.864969 9977 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:42255
I20260619 12:24:37.865240 10044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:42255 every 8 connection(s)
I20260619 12:24:37.866070 9977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:37.866433 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 9977
I20260619 12:24:37.866587 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:37.866907 7071 external_mini_cluster.cc:1428] Setting key 8648c8c7344150cc1d40a42e87b84e05
I20260619 12:24:37.869212 10045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:37.875231 10045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea: Bootstrap starting.
Jun 19 12:24:37 dist-test-slave-kr9h krb5kdc[9964](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:37.877372 10045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:37.878108 10045 log.cc:824] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:37.880441 10045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea: No bootstrap required, opened a new log
I20260619 12:24:37.881157 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:37.868301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.6.231.254:42255)
0619 12:24:37.868721 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:37.868744 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:37.868780 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:24:37.869539 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:37.869557 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:37.869887 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:37.870173 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:37.871046 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:37.872123 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:37.873244 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:37.873815 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:37.873996 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:37.876474 (+ 2478us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:37.876480 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:37.876492 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:37.876518 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:37.878195 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:37.878704 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:37.878713 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:37.878721 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:37.878865 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:37.879217 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:37.879220 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:37.879221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:37.879597 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:37.879741 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:37.880051 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 12:24:37.880383 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":155,"threads_started":1}
I20260619 12:24:37.884124 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:24:37.884613 10045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fcd9df24054f9bbce9c203ce136dea" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42255 } }
I20260619 12:24:37.884869 10045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:37.884971 10045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 04fcd9df24054f9bbce9c203ce136dea, State: Initialized, Role: FOLLOWER
I20260619 12:24:37.885486 10045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fcd9df24054f9bbce9c203ce136dea" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42255 } }
I20260619 12:24:37.885646 10045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:37.885713 10045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:37.885816 10045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:37.887279 10045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fcd9df24054f9bbce9c203ce136dea" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42255 } }
I20260619 12:24:37.887655 10045 leader_election.cc:302] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 04fcd9df24054f9bbce9c203ce136dea; no voters:
I20260619 12:24:37.888023 10045 leader_election.cc:288] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:37.888109 10051 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:37.888299 10051 raft_consensus.cc:686] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [term 1 LEADER]: Becoming Leader. State: Replica: 04fcd9df24054f9bbce9c203ce136dea, State: Running, Role: LEADER
I20260619 12:24:37.888618 10051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fcd9df24054f9bbce9c203ce136dea" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42255 } }
I20260619 12:24:37.889372 10045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:37.889829 10053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 04fcd9df24054f9bbce9c203ce136dea. Latest consensus state: current_term: 1 leader_uuid: "04fcd9df24054f9bbce9c203ce136dea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fcd9df24054f9bbce9c203ce136dea" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42255 } } }
I20260619 12:24:37.889966 10053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:37.890367 10056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:37.891376 10052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "04fcd9df24054f9bbce9c203ce136dea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "04fcd9df24054f9bbce9c203ce136dea" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42255 } } }
I20260619 12:24:37.891482 10052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:37.893800 10056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:37.899974 10056 catalog_manager.cc:1347] Generated new cluster ID: ee07ba9d0c8c4e6c99c65a2fff3ee112
I20260619 12:24:37.900043 10056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:37.903512 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:37.898673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34118 (local address 127.6.231.254:42255)
0619 12:24:37.898782 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:37.898786 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:37.899041 (+ 255us) server_negotiation.cc:407] Connection header received
0619 12:24:37.899197 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:37.899200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:37.899250 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:37.899329 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:37.899891 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:37.900496 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:37.901311 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:37.901645 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:37.901756 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:37.902627 (+ 871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:37.902630 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:37.902633 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:37.903057 (+ 424us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:37.903085 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:37.903100 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:37.903200 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:37.903310 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 12:24:37.903368 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260619 12:24:37.918987 10056 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:37.919952 10056 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:37.931684 10056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 04fcd9df24054f9bbce9c203ce136dea: Generated new TSK 0
I20260619 12:24:37.932431 10056 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:24:37.942742 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 9977
2026-06-19T12:24:37Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1413 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:24:37 dist-test-slave-kr9h krb5kdc[10078](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:24:37 dist-test-slave-kr9h krb5kdc[10078](info): set up 2 sockets
Jun 19 12:24:37 dist-test-slave-kr9h krb5kdc[10078](info): commencing operation
krb5kdc: starting...
W20260619 12:24:38.989550 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:24:39 dist-test-slave-kr9h krb5kdc[10078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:24:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:24:39.116268 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:35085
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:46335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:35085
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:39.229322 10091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:39.229736 10091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:39.229822 10091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:39.234131 10091 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:39.234211 10091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:39.234248 10091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:39.234318 10091 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:39.234355 10091 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:39.241839 10091 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:46335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:35085
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:35085
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:39.243152 10091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:39.244232 10091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:39.253535 10097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:39.253530 10096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:39.253628 10091 server_base.cc:1034] running on GCE node
W20260619 12:24:39.253530 10099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:39.254493 10091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:39.255725 10091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:39.256927 10091 hybrid_clock.cc:648] HybridClock initialized: now 1781871879256902 us; error 38 us; skew 500 ppm
Jun 19 12:24:39 dist-test-slave-kr9h krb5kdc[10078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871879, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:39.260154 10091 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:24:39.261328 10091 webserver.cc:460] Webserver started at http://127.6.231.254:45595/ using document root <none> and password file <none>
I20260619 12:24:39.261950 10091 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:39.262048 10091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:39.262293 10091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:39.264339 10091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "9df550d33d2d40cfa58a0fd7e480c6df"
format_stamp: "Formatted at 2026-06-19 12:24:39 on dist-test-slave-kr9h"
server_key: "a4cc03955ab36988c21f6cc673d681ae"
server_key_iv: "6bd0b6bbb3582b24187d93584c5a6511"
server_key_version: "encryptionkey@0"
I20260619 12:24:39.264906 10091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "9df550d33d2d40cfa58a0fd7e480c6df"
format_stamp: "Formatted at 2026-06-19 12:24:39 on dist-test-slave-kr9h"
server_key: "a4cc03955ab36988c21f6cc673d681ae"
server_key_iv: "6bd0b6bbb3582b24187d93584c5a6511"
server_key_version: "encryptionkey@0"
I20260619 12:24:39.268774 10091 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 12:24:39.271235 10106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:39.272485 10091 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 12:24:39.272622 10091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "9df550d33d2d40cfa58a0fd7e480c6df"
format_stamp: "Formatted at 2026-06-19 12:24:39 on dist-test-slave-kr9h"
server_key: "a4cc03955ab36988c21f6cc673d681ae"
server_key_iv: "6bd0b6bbb3582b24187d93584c5a6511"
server_key_version: "encryptionkey@0"
I20260619 12:24:39.272732 10091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:39.286531 10091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:39.289371 10091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:39.289585 10091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:39.296228 10091 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:35085
I20260619 12:24:39.296541 10158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:35085 every 8 connection(s)
I20260619 12:24:39.297192 10091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:39.299916 10159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:39.302642 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 10091
I20260619 12:24:39.302783 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:39.303056 7071 external_mini_cluster.cc:1428] Setting key 8ee629bf709943a2e83546ec59fcab84
I20260619 12:24:39.306053 10159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df: Bootstrap starting.
I20260619 12:24:39.308768 10159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:39.309590 10159 log.cc:824] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:39.311841 10159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df: No bootstrap required, opened a new log
Jun 19 12:24:39 dist-test-slave-kr9h krb5kdc[10078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871879, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:39.315810 10159 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df550d33d2d40cfa58a0fd7e480c6df" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35085 } }
I20260619 12:24:39.316032 10159 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:39.316067 10159 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9df550d33d2d40cfa58a0fd7e480c6df, State: Initialized, Role: FOLLOWER
I20260619 12:24:39.316615 10159 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df550d33d2d40cfa58a0fd7e480c6df" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35085 } }
I20260619 12:24:39.316794 10159 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:39.316869 10159 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:39.316987 10159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:39.318202 10159 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df550d33d2d40cfa58a0fd7e480c6df" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35085 } }
I20260619 12:24:39.318596 10159 leader_election.cc:302] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9df550d33d2d40cfa58a0fd7e480c6df; no voters:
I20260619 12:24:39.318989 10159 leader_election.cc:288] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:39.319135 10164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:39.319413 10164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [term 1 LEADER]: Becoming Leader. State: Replica: 9df550d33d2d40cfa58a0fd7e480c6df, State: Running, Role: LEADER
I20260619 12:24:39.319777 10164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df550d33d2d40cfa58a0fd7e480c6df" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35085 } }
I20260619 12:24:39.319926 10162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:39.304391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44460 (local address 127.6.231.254:35085)
0619 12:24:39.304838 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:39.304857 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:39.304900 (+ 43us) server_negotiation.cc:407] Connection header received
0619 12:24:39.305735 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:39.305759 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:39.306178 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:39.306549 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:39.307326 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:39.308790 (+ 1464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:39.309895 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:39.310491 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:39.310645 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:39.313941 (+ 3296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:39.313946 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:39.313960 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:39.313985 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:39.316513 (+ 2528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:39.317041 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:39.317050 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:39.317055 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:39.317201 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:39.317565 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:39.317568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:39.317570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:39.318089 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:39.318283 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:39.318669 (+ 386us) server_negotiation.cc:299] Negotiation successful
0619 12:24:39.318981 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":123,"threads_started":1}
I20260619 12:24:39.320232 10159 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:39.322548 10165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9df550d33d2d40cfa58a0fd7e480c6df. Latest consensus state: current_term: 1 leader_uuid: "9df550d33d2d40cfa58a0fd7e480c6df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df550d33d2d40cfa58a0fd7e480c6df" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35085 } } }
I20260619 12:24:39.322665 10165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:39.322845 10166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9df550d33d2d40cfa58a0fd7e480c6df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9df550d33d2d40cfa58a0fd7e480c6df" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35085 } } }
I20260619 12:24:39.323192 10166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df [sys.catalog]: This master's current role is: LEADER
W20260619 12:24:39.325598 10178 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:24:39.325677 10178 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:24:39.325762 10179 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:39.326453 10179 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:39.331784 10179 catalog_manager.cc:1347] Generated new cluster ID: 08d7462d7795450b9cd7ac61057855ed
I20260619 12:24:39.331866 10179 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:39.351831 10179 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:39.352794 10179 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:39.360494 10179 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9df550d33d2d40cfa58a0fd7e480c6df: Generated new TSK 0
I20260619 12:24:39.361224 10179 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:24:39.390718 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35085
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:46335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:39.504364 10185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:39.504621 10185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:39.504664 10185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:39.508769 10185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:39.508836 10185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:39.508894 10185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:24:39.514036 10185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:46335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:39.515071 10185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:39.516124 10185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:39.526361 10191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:39.526356 10190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:39.526356 10193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:39.526818 10185 server_base.cc:1034] running on GCE node
I20260619 12:24:39.527251 10185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:39.527899 10185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:39.529088 10185 hybrid_clock.cc:648] HybridClock initialized: now 1781871879529066 us; error 33 us; skew 500 ppm
Jun 19 12:24:39 dist-test-slave-kr9h krb5kdc[10078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871879, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:39.538021 10185 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:24:39.539002 10185 webserver.cc:460] Webserver started at http://127.6.231.193:32775/ using document root <none> and password file <none>
I20260619 12:24:39.539583 10185 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:39.539675 10185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:39.539912 10185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:39.542160 10185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "0057aaa79eb746b3afd63730bdef1ab5"
format_stamp: "Formatted at 2026-06-19 12:24:39 on dist-test-slave-kr9h"
server_key: "43f5aed067df29f4e2fd3f8dbf20408f"
server_key_iv: "276647b2d470f52dcb4e82c242a938eb"
server_key_version: "encryptionkey@0"
I20260619 12:24:39.542742 10185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "0057aaa79eb746b3afd63730bdef1ab5"
format_stamp: "Formatted at 2026-06-19 12:24:39 on dist-test-slave-kr9h"
server_key: "43f5aed067df29f4e2fd3f8dbf20408f"
server_key_iv: "276647b2d470f52dcb4e82c242a938eb"
server_key_version: "encryptionkey@0"
I20260619 12:24:39.546526 10185 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 12:24:39.549052 10200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:39.550421 10185 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:24:39.550568 10185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "0057aaa79eb746b3afd63730bdef1ab5"
format_stamp: "Formatted at 2026-06-19 12:24:39 on dist-test-slave-kr9h"
server_key: "43f5aed067df29f4e2fd3f8dbf20408f"
server_key_iv: "276647b2d470f52dcb4e82c242a938eb"
server_key_version: "encryptionkey@0"
I20260619 12:24:39.550684 10185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:39.564606 10185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:39.567754 10185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:39.567960 10185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:39.568569 10185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:39.569375 10185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:39.569418 10185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:39.569563 10185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:39.569595 10185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:39.578619 10185 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:43229
I20260619 12:24:39.578691 10313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:43229 every 8 connection(s)
I20260619 12:24:39.579559 10185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:24:39.587275 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 10185
I20260619 12:24:39.587445 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:24:39.587785 7071 external_mini_cluster.cc:1428] Setting key 69df84fa4df503dec8d715a7950a6aa5
Jun 19 12:24:39 dist-test-slave-kr9h krb5kdc[10078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871879, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:39.593948 10162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:39.581508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:35251 (local address 127.6.231.254:35085)
0619 12:24:39.581641 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:39.581644 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:39.582591 (+ 947us) server_negotiation.cc:407] Connection header received
0619 12:24:39.583953 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:39.583956 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:39.584009 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:39.584096 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:39.585268 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:39.585888 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:39.587039 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:39.587277 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:39.587390 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:39.590045 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:39.590048 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:39.590050 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:39.590072 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:39.591548 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:39.592170 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:39.592173 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:39.592174 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:39.592263 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:39.592673 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:39.592676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:39.592677 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:39.592886 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:39.592970 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:39.593685 (+ 715us) server_negotiation.cc:299] Negotiation successful
0619 12:24:39.593799 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260619 12:24:39.595203 10316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:39.581974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35085 (local address 127.6.231.193:35251)
0619 12:24:39.582430 (+ 456us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:39.582479 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:39.583745 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:39.584278 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:39.584285 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:39.584632 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:39.585120 (+ 488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:39.585132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:39.586028 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:39.586032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:39.586901 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:39.586910 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:39.587428 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:39.587432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:39.587647 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:39.588297 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:39.588313 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:39.589876 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:39.591682 (+ 1806us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:39.591688 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:39.591703 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:39.592062 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:39.592367 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:39.592373 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:39.592379 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:39.592575 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:39.592997 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:39.593006 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:39.593350 (+ 344us) client_negotiation.cc:769] Sending connection context
0619 12:24:39.593624 (+ 274us) client_negotiation.cc:240] Negotiation successful
0619 12:24:39.594001 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":126,"threads_started":1}
I20260619 12:24:39.596658 10314 heartbeater.cc:347] Connected to a master server at 127.6.231.254:35085
I20260619 12:24:39.596936 10314 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:39.597571 10314 heartbeater.cc:511] Master 127.6.231.254:35085 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
I20260619 12:24:39.599247 10123 ts_manager.cc:194] Registered new tserver with Master: 0057aaa79eb746b3afd63730bdef1ab5 (127.6.231.193:43229)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:24:39.600685 10123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:35251
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:24:39.616200 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35085
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:46335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:39.730758 10319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:39.731092 10319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:39.731169 10319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:39.735225 10319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:39.735339 10319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:39.735440 10319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:24:39.740815 10319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:46335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:39.741964 10319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:39.743062 10319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:39.752712 10324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:39.752796 10319 server_base.cc:1034] running on GCE node
W20260619 12:24:39.752688 10327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:39.752748 10325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:39.753419 10319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:39.754230 10319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:39.755429 10319 hybrid_clock.cc:648] HybridClock initialized: now 1781871879755386 us; error 60 us; skew 500 ppm
Jun 19 12:24:39 dist-test-slave-kr9h krb5kdc[10078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871879, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:39.758154 10319 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:24:39.759016 10319 webserver.cc:460] Webserver started at http://127.6.231.194:41543/ using document root <none> and password file <none>
I20260619 12:24:39.759620 10319 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:39.759724 10319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:39.759954 10319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:39.762073 10319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "c3db1c7178174ec0960818dfcd497d8f"
format_stamp: "Formatted at 2026-06-19 12:24:39 on dist-test-slave-kr9h"
server_key: "bd5b0c583743648755d943ebc9af3420"
server_key_iv: "9b65576a27eac28be709ba610ac28567"
server_key_version: "encryptionkey@0"
I20260619 12:24:39.762678 10319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "c3db1c7178174ec0960818dfcd497d8f"
format_stamp: "Formatted at 2026-06-19 12:24:39 on dist-test-slave-kr9h"
server_key: "bd5b0c583743648755d943ebc9af3420"
server_key_iv: "9b65576a27eac28be709ba610ac28567"
server_key_version: "encryptionkey@0"
I20260619 12:24:39.766435 10319 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 12:24:39.768965 10334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:39.770256 10319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 12:24:39.770440 10319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "c3db1c7178174ec0960818dfcd497d8f"
format_stamp: "Formatted at 2026-06-19 12:24:39 on dist-test-slave-kr9h"
server_key: "bd5b0c583743648755d943ebc9af3420"
server_key_iv: "9b65576a27eac28be709ba610ac28567"
server_key_version: "encryptionkey@0"
I20260619 12:24:39.770558 10319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:39.782790 10319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:39.785615 10319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:39.785810 10319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:39.786389 10319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:39.787178 10319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:39.787251 10319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:39.787317 10319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:39.787376 10319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:39.796131 10319 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:40613
I20260619 12:24:39.796154 10447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:40613 every 8 connection(s)
I20260619 12:24:39.797046 10319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:24:39.803086 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 10319
I20260619 12:24:39.803243 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:24:39.803534 7071 external_mini_cluster.cc:1428] Setting key 977126721d694ead7ff369c1e3851e0a
Jun 19 12:24:39 dist-test-slave-kr9h krb5kdc[10078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871879, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:39.811329 10162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:39.798877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:35879 (local address 127.6.231.254:35085)
0619 12:24:39.799032 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:39.799036 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:39.799786 (+ 750us) server_negotiation.cc:407] Connection header received
0619 12:24:39.801162 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:39.801165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:39.801226 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:39.801339 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:39.802562 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:39.803122 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:39.804508 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:39.804824 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:39.804946 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:39.807525 (+ 2579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:39.807528 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:39.807531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:39.807556 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:39.809076 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:39.809732 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:39.809736 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:39.809738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:39.809824 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:39.810199 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:39.810203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:39.810204 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:39.810397 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:39.810508 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:39.811038 (+ 530us) server_negotiation.cc:299] Negotiation successful
0619 12:24:39.811166 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 12:24:39.812181 10450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:39.799199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35085 (local address 127.6.231.194:35879)
0619 12:24:39.799635 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:39.799676 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:39.800935 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:39.801544 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:39.801552 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:39.801917 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:39.802406 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:39.802422 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:39.803259 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:39.803263 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:39.804385 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:39.804394 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:39.804986 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:39.804990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:39.805166 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:39.805837 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:39.805855 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:39.807332 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:39.809222 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:39.809228 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:39.809246 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:39.809624 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:39.809925 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:39.809931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:39.809935 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:39.810092 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:39.810516 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:39.810524 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:39.810788 (+ 264us) client_negotiation.cc:769] Sending connection context
0619 12:24:39.810983 (+ 195us) client_negotiation.cc:240] Negotiation successful
0619 12:24:39.811320 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":120,"threads_started":1}
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
I20260619 12:24:39.813560 10448 heartbeater.cc:347] Connected to a master server at 127.6.231.254:35085
I20260619 12:24:39.813817 10448 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:39.814342 10448 heartbeater.cc:511] Master 127.6.231.254:35085 requested a full tablet report, sending...
I20260619 12:24:39.815455 10123 ts_manager.cc:194] Registered new tserver with Master: c3db1c7178174ec0960818dfcd497d8f (127.6.231.194:40613)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
I20260619 12:24:39.816092 10123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:35879
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:24:39.832870 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35085
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:46335
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:39.945784 10453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:39.946069 10453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:39.946115 10453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:39.950616 10453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:39.950678 10453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:39.950747 10453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:24:39.956282 10453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:46335
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35085
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:39.957355 10453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:39.958454 10453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:39.968233 10459 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:39.968232 10458 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:39.968361 10461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:39.968549 10453 server_base.cc:1034] running on GCE node
I20260619 12:24:39.969148 10453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:39.969867 10453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:39.971060 10453 hybrid_clock.cc:648] HybridClock initialized: now 1781871879971028 us; error 49 us; skew 500 ppm
Jun 19 12:24:39 dist-test-slave-kr9h krb5kdc[10078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871879, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:39.973768 10453 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:24:39.974625 10453 webserver.cc:460] Webserver started at http://127.6.231.195:36773/ using document root <none> and password file <none>
I20260619 12:24:39.975136 10453 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:39.975193 10453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:39.975350 10453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:39.977373 10453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "9d6183b3b193466087b3980e078b1e85"
format_stamp: "Formatted at 2026-06-19 12:24:39 on dist-test-slave-kr9h"
server_key: "2b5e0617374d489978341b1d99a59313"
server_key_iv: "2f87da7f6a435b920113fbd2864bf277"
server_key_version: "encryptionkey@0"
I20260619 12:24:39.977945 10453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "9d6183b3b193466087b3980e078b1e85"
format_stamp: "Formatted at 2026-06-19 12:24:39 on dist-test-slave-kr9h"
server_key: "2b5e0617374d489978341b1d99a59313"
server_key_iv: "2f87da7f6a435b920113fbd2864bf277"
server_key_version: "encryptionkey@0"
I20260619 12:24:39.981673 10453 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 12:24:39.984069 10468 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:39.985532 10453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:39.985652 10453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "9d6183b3b193466087b3980e078b1e85"
format_stamp: "Formatted at 2026-06-19 12:24:39 on dist-test-slave-kr9h"
server_key: "2b5e0617374d489978341b1d99a59313"
server_key_iv: "2f87da7f6a435b920113fbd2864bf277"
server_key_version: "encryptionkey@0"
I20260619 12:24:39.985767 10453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:40.006232 10453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:40.009780 10453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:40.009995 10453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:40.010620 10453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:40.011441 10453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:40.011488 10453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:40.011528 10453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:40.011543 10453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:40.021005 10453 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:40987
I20260619 12:24:40.021026 10581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:40987 every 8 connection(s)
I20260619 12:24:40.022018 10453 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:24:40.029516 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 10453
I20260619 12:24:40.029613 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:24:40.029920 7071 external_mini_cluster.cc:1428] Setting key 01742c3d1d6762b3521e3137b38fb939
Jun 19 12:24:40 dist-test-slave-kr9h krb5kdc[10078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871879, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:40.036556 10162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:40.023912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:36407 (local address 127.6.231.254:35085)
0619 12:24:40.024056 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:40.024068 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:40.024864 (+ 796us) server_negotiation.cc:407] Connection header received
0619 12:24:40.026335 (+ 1471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:40.026338 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:40.026406 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:40.026514 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:40.027714 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:40.028251 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:40.029419 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:40.029764 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:40.029881 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:40.032801 (+ 2920us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:40.032804 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:40.032807 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:40.032833 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:40.034438 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:40.034990 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:40.034995 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:40.034997 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:40.035081 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:40.035437 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:40.035442 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:40.035443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:40.035633 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:40.035744 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:40.036259 (+ 515us) server_negotiation.cc:299] Negotiation successful
0619 12:24:40.036378 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:24:40.037352 10584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:40.024254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35085 (local address 127.6.231.195:36407)
0619 12:24:40.024702 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:40.024745 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:40.026105 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:40.026694 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:40.026702 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:40.027080 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:40.027558 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:40.027571 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:40.028380 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:40.028384 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:40.029278 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:40.029287 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:40.029931 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:40.029935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:40.030147 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:40.030787 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:40.030849 (+ 62us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:40.032631 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:40.034569 (+ 1938us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:40.034576 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:40.034597 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:40.034864 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:40.035182 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:40.035185 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:40.035187 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:40.035333 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:40.035742 (+ 409us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:40.035749 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:40.036012 (+ 263us) client_negotiation.cc:769] Sending connection context
0619 12:24:40.036209 (+ 197us) client_negotiation.cc:240] Negotiation successful
0619 12:24:40.036537 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":126,"threads_started":1}
I20260619 12:24:40.038661 10582 heartbeater.cc:347] Connected to a master server at 127.6.231.254:35085
I20260619 12:24:40.039014 10582 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:40.039595 10582 heartbeater.cc:511] Master 127.6.231.254:35085 requested a full tablet report, sending...
I20260619 12:24:40.040915 10123 ts_manager.cc:194] Registered new tserver with Master: 9d6183b3b193466087b3980e078b1e85 (127.6.231.195:40987)
I20260619 12:24:40.041704 10123 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:36407
I20260619 12:24:40.044307 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:24:40 dist-test-slave-kr9h krb5kdc[10078](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871880, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Jun 19 12:24:40 dist-test-slave-kr9h krb5kdc[10078](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871880, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:40.071283 10162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:40.062469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40350 (local address 127.6.231.254:35085)
0619 12:24:40.062620 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:40.062624 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:40.062835 (+ 211us) server_negotiation.cc:407] Connection header received
0619 12:24:40.062953 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:40.062955 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:40.063000 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:40.063071 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:40.063849 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:40.064355 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:40.065238 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:40.065506 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:40.065585 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:40.068000 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:40.068003 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:40.068005 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:40.068028 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:40.069682 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:40.070068 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:40.070071 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:40.070072 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:40.070142 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:40.070427 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:40.070430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:40.070431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:40.070632 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:40.070733 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:40.070994 (+ 261us) server_negotiation.cc:299] Negotiation successful
0619 12:24:40.071086 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260619 12:24:40.071741 10123 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:40350
I20260619 12:24:40.072624 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 10185
I20260619 12:24:40.078768 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 10319
I20260619 12:24:40.084036 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 10453
I20260619 12:24:40.089248 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 10091
2026-06-19T12:24:40Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2149 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:24:40 dist-test-slave-kr9h krb5kdc[10594](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:24:40 dist-test-slave-kr9h krb5kdc[10594](info): set up 2 sockets
Jun 19 12:24:40 dist-test-slave-kr9h krb5kdc[10594](info): commencing operation
krb5kdc: starting...
W20260619 12:24:41.141985 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:24:41 dist-test-slave-kr9h krb5kdc[10594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:24:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:24:41.270557 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:34917
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:46525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:34917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:41.378983 10607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:41.379241 10607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:41.379279 10607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:41.383323 10607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:41.383391 10607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:41.383410 10607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:41.383427 10607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:41.383496 10607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:41.388738 10607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:46525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:34917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:34917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:41.389889 10607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:41.390969 10607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:41.399663 10612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:41.399747 10607 server_base.cc:1034] running on GCE node
W20260619 12:24:41.399677 10615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:41.399677 10613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:41.400470 10607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:41.401734 10607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:41.402956 10607 hybrid_clock.cc:648] HybridClock initialized: now 1781871881402941 us; error 28 us; skew 500 ppm
Jun 19 12:24:41 dist-test-slave-kr9h krb5kdc[10594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871881, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:41.406031 10607 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:24:41.406975 10607 webserver.cc:460] Webserver started at http://127.6.231.254:42977/ using document root <none> and password file <none>
I20260619 12:24:41.407548 10607 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:41.407608 10607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:41.407829 10607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:41.409996 10607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "da5e2675bc6c48239042a9f72931c848"
format_stamp: "Formatted at 2026-06-19 12:24:41 on dist-test-slave-kr9h"
server_key: "0ccdc18bc82a0fa3878a69aeb3e7f7e5"
server_key_iv: "2d72a8766896598014471adff8f1d897"
server_key_version: "encryptionkey@0"
I20260619 12:24:41.410585 10607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "da5e2675bc6c48239042a9f72931c848"
format_stamp: "Formatted at 2026-06-19 12:24:41 on dist-test-slave-kr9h"
server_key: "0ccdc18bc82a0fa3878a69aeb3e7f7e5"
server_key_iv: "2d72a8766896598014471adff8f1d897"
server_key_version: "encryptionkey@0"
I20260619 12:24:41.414374 10607 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 12:24:41.416810 10622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:41.418128 10607 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:24:41.418259 10607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "da5e2675bc6c48239042a9f72931c848"
format_stamp: "Formatted at 2026-06-19 12:24:41 on dist-test-slave-kr9h"
server_key: "0ccdc18bc82a0fa3878a69aeb3e7f7e5"
server_key_iv: "2d72a8766896598014471adff8f1d897"
server_key_version: "encryptionkey@0"
I20260619 12:24:41.418365 10607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:41.445794 10607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:41.449337 10607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:41.449581 10607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:41.456658 10607 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:34917
I20260619 12:24:41.456694 10674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:34917 every 8 connection(s)
I20260619 12:24:41.457648 10607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:41.460762 10675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:41.466871 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 10607
I20260619 12:24:41.467015 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:41.467123 10675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848: Bootstrap starting.
I20260619 12:24:41.467319 7071 external_mini_cluster.cc:1428] Setting key 26e7eba1e2002589ada0438499cdddcf
I20260619 12:24:41.470170 10675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:41.471143 10675 log.cc:824] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:41.473143 10675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848: No bootstrap required, opened a new log
Jun 19 12:24:41 dist-test-slave-kr9h krb5kdc[10594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871881, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:41.477552 10675 raft_consensus.cc:348] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5e2675bc6c48239042a9f72931c848" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34917 } }
I20260619 12:24:41.477789 10675 raft_consensus.cc:374] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:41.477865 10675 raft_consensus.cc:729] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da5e2675bc6c48239042a9f72931c848, State: Initialized, Role: FOLLOWER
I20260619 12:24:41.478358 10675 consensus_queue.cc:260] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5e2675bc6c48239042a9f72931c848" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34917 } }
I20260619 12:24:41.478506 10675 raft_consensus.cc:388] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:41.478590 10675 raft_consensus.cc:482] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:41.478684 10675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:41.479565 10675 raft_consensus.cc:504] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5e2675bc6c48239042a9f72931c848" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34917 } }
I20260619 12:24:41.479909 10675 leader_election.cc:302] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da5e2675bc6c48239042a9f72931c848; no voters:
I20260619 12:24:41.480221 10675 leader_election.cc:288] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:41.480371 10680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:41.480587 10680 raft_consensus.cc:686] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [term 1 LEADER]: Becoming Leader. State: Replica: da5e2675bc6c48239042a9f72931c848, State: Running, Role: LEADER
I20260619 12:24:41.480942 10680 consensus_queue.cc:237] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5e2675bc6c48239042a9f72931c848" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34917 } }
I20260619 12:24:41.481839 10675 sys_catalog.cc:564] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:41.482744 10681 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da5e2675bc6c48239042a9f72931c848" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5e2675bc6c48239042a9f72931c848" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34917 } } }
I20260619 12:24:41.482867 10681 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:41.483271 10682 sys_catalog.cc:455] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [sys.catalog]: SysCatalogTable state changed. Reason: New leader da5e2675bc6c48239042a9f72931c848. Latest consensus state: current_term: 1 leader_uuid: "da5e2675bc6c48239042a9f72931c848" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da5e2675bc6c48239042a9f72931c848" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34917 } } }
I20260619 12:24:41.483348 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:41.468659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40318 (local address 127.6.231.254:34917)
0619 12:24:41.469102 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:41.469122 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:41.469168 (+ 46us) server_negotiation.cc:407] Connection header received
0619 12:24:41.470109 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:41.470137 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:41.470623 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:41.471022 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:41.471782 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:41.472938 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:41.474213 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:41.474674 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:41.474800 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:41.477773 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:41.477778 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:41.477819 (+ 41us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:41.477849 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:41.479895 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:41.480432 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:41.480442 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:41.480451 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:41.480598 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:41.480933 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:41.480937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:41.480938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:41.481470 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:41.481666 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:41.481972 (+ 306us) server_negotiation.cc:299] Negotiation successful
0619 12:24:41.482298 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":118,"threads_started":1}
I20260619 12:24:41.483424 10682 sys_catalog.cc:458] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848 [sys.catalog]: This master's current role is: LEADER
W20260619 12:24:41.486829 10696 catalog_manager.cc:1558] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:24:41.486907 10696 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:24:41.487035 10689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:41.487911 10689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:41.493848 10689 catalog_manager.cc:1347] Generated new cluster ID: dd29305cb6324e8291aebf26862e5894
I20260619 12:24:41.493959 10689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:41.510962 10689 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:41.512133 10689 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:41.523643 10689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da5e2675bc6c48239042a9f72931c848: Generated new TSK 0
I20260619 12:24:41.524462 10689 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:24:41.564424 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:34917
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:46525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:41.677848 10701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:41.678113 10701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:41.678155 10701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:41.682386 10701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:41.682447 10701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:41.682507 10701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:24:41.687942 10701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:46525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:34917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:41.689056 10701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:41.690205 10701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:41.700073 10709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:41.700114 10706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:41.700177 10701 server_base.cc:1034] running on GCE node
W20260619 12:24:41.700193 10707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:41.700721 10701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:41.701323 10701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:41.702541 10701 hybrid_clock.cc:648] HybridClock initialized: now 1781871881702500 us; error 66 us; skew 500 ppm
Jun 19 12:24:41 dist-test-slave-kr9h krb5kdc[10594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871881, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:41.705358 10701 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:24:41.706288 10701 webserver.cc:460] Webserver started at http://127.6.231.193:45529/ using document root <none> and password file <none>
I20260619 12:24:41.706854 10701 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:41.706957 10701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:41.707254 10701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:41.709316 10701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "f45f6e18ec5442a893efd685b8e61f19"
format_stamp: "Formatted at 2026-06-19 12:24:41 on dist-test-slave-kr9h"
server_key: "4af7d0e25ea38d06c4b9a69efc5bb645"
server_key_iv: "bfb5a2de825455e11b2abf3c6bdbd4a7"
server_key_version: "encryptionkey@0"
I20260619 12:24:41.709949 10701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "f45f6e18ec5442a893efd685b8e61f19"
format_stamp: "Formatted at 2026-06-19 12:24:41 on dist-test-slave-kr9h"
server_key: "4af7d0e25ea38d06c4b9a69efc5bb645"
server_key_iv: "bfb5a2de825455e11b2abf3c6bdbd4a7"
server_key_version: "encryptionkey@0"
I20260619 12:24:41.714037 10701 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:24:41.716924 10716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:41.718384 10701 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:41.718518 10701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "f45f6e18ec5442a893efd685b8e61f19"
format_stamp: "Formatted at 2026-06-19 12:24:41 on dist-test-slave-kr9h"
server_key: "4af7d0e25ea38d06c4b9a69efc5bb645"
server_key_iv: "bfb5a2de825455e11b2abf3c6bdbd4a7"
server_key_version: "encryptionkey@0"
I20260619 12:24:41.718638 10701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:41.729807 10701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:41.733273 10701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:41.733523 10701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:41.734126 10701 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:41.735002 10701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:41.735047 10701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:41.735085 10701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:41.735143 10701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:41.744549 10701 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:46463
I20260619 12:24:41.744581 10829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:46463 every 8 connection(s)
I20260619 12:24:41.745561 10701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:24:41.751032 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 10701
I20260619 12:24:41.751253 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:24:41.751614 7071 external_mini_cluster.cc:1428] Setting key 60ddfac87489a72cee938cb4d6719c6f
Jun 19 12:24:41 dist-test-slave-kr9h krb5kdc[10594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871881, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
I20260619 12:24:41.761752 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:41.747451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:49727 (local address 127.6.231.254:34917)
0619 12:24:41.747647 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:41.747652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:41.748499 (+ 847us) server_negotiation.cc:407] Connection header received
0619 12:24:41.750016 (+ 1517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:41.750020 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:41.750071 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:41.750155 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:41.751519 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:41.752274 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:41.753873 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:41.754295 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:41.754420 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:41.757479 (+ 3059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:41.757483 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:41.757487 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:41.757519 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:41.759334 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:41.759905 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:41.759910 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:41.759914 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:41.760012 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:41.760368 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:41.760371 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:41.760373 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:41.760632 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:41.760746 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:41.761314 (+ 568us) server_negotiation.cc:299] Negotiation successful
0619 12:24:41.761527 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 12:24:41.762629 10832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:41.747864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34917 (local address 127.6.231.193:49727)
0619 12:24:41.748331 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:41.748375 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:41.749778 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:41.750329 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:41.750336 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:41.750729 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:41.751297 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:41.751321 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:41.752416 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:41.752420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:41.753743 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:41.753753 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:41.754500 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:41.754503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:41.754697 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:41.755551 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:41.755576 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:41.757251 (+ 1675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:41.759478 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:41.759484 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:41.759499 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:41.759787 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:41.760127 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:41.760129 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:41.760130 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:41.760267 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:41.760750 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:41.760756 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:41.761002 (+ 246us) client_negotiation.cc:769] Sending connection context
0619 12:24:41.761231 (+ 229us) client_negotiation.cc:240] Negotiation successful
0619 12:24:41.761621 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":125,"threads_started":1}
I20260619 12:24:41.764065 10830 heartbeater.cc:347] Connected to a master server at 127.6.231.254:34917
I20260619 12:24:41.764313 10830 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:24:41.764947 10830 heartbeater.cc:511] Master 127.6.231.254:34917 requested a full tablet report, sending...
I20260619 12:24:41.766883 10636 ts_manager.cc:194] Registered new tserver with Master: f45f6e18ec5442a893efd685b8e61f19 (127.6.231.193:46463)
I20260619 12:24:41.768755 10636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:49727
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:24:41.784236 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:34917
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:46525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:41.898201 10835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:41.898458 10835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:41.898501 10835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:41.902796 10835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:41.902859 10835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:41.902920 10835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:24:41.908299 10835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:46525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:34917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:41.909288 10835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:41.910436 10835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:41.919935 10843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:41.919938 10840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:41.919940 10841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:41.920228 10835 server_base.cc:1034] running on GCE node
I20260619 12:24:41.920728 10835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:41.921497 10835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:41.922691 10835 hybrid_clock.cc:648] HybridClock initialized: now 1781871881922669 us; error 43 us; skew 500 ppm
Jun 19 12:24:41 dist-test-slave-kr9h krb5kdc[10594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871881, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:41.925770 10835 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:24:41.926659 10835 webserver.cc:460] Webserver started at http://127.6.231.194:37427/ using document root <none> and password file <none>
I20260619 12:24:41.927175 10835 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:41.927232 10835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:41.927397 10835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:41.929348 10835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "30067b4cd3d84a5eb5a0b86b95566fba"
format_stamp: "Formatted at 2026-06-19 12:24:41 on dist-test-slave-kr9h"
server_key: "738d7b0d3ed6c2a3ecf570a968dda7b8"
server_key_iv: "a59555260a4c6c76b5fe9330ad029a60"
server_key_version: "encryptionkey@0"
I20260619 12:24:41.929940 10835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "30067b4cd3d84a5eb5a0b86b95566fba"
format_stamp: "Formatted at 2026-06-19 12:24:41 on dist-test-slave-kr9h"
server_key: "738d7b0d3ed6c2a3ecf570a968dda7b8"
server_key_iv: "a59555260a4c6c76b5fe9330ad029a60"
server_key_version: "encryptionkey@0"
I20260619 12:24:41.933647 10835 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:24:41.936043 10850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:41.937623 10835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:41.937767 10835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "30067b4cd3d84a5eb5a0b86b95566fba"
format_stamp: "Formatted at 2026-06-19 12:24:41 on dist-test-slave-kr9h"
server_key: "738d7b0d3ed6c2a3ecf570a968dda7b8"
server_key_iv: "a59555260a4c6c76b5fe9330ad029a60"
server_key_version: "encryptionkey@0"
I20260619 12:24:41.937893 10835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:41.956822 10835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:41.960134 10835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:41.960352 10835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:41.961015 10835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:41.961922 10835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:41.961971 10835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:41.962070 10835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:41.962113 10835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:41.972152 10835 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:43567
I20260619 12:24:41.972187 10963 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:43567 every 8 connection(s)
I20260619 12:24:41.973140 10835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:24:41.981072 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 10835
I20260619 12:24:41.981261 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:24:41.981624 7071 external_mini_cluster.cc:1428] Setting key 59a7512714fce889c6df5a8342f78d92
Jun 19 12:24:41 dist-test-slave-kr9h krb5kdc[10594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871881, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:41.988746 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:41.975278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:48949 (local address 127.6.231.254:34917)
0619 12:24:41.975422 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:41.975426 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:41.976294 (+ 868us) server_negotiation.cc:407] Connection header received
0619 12:24:41.977706 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:41.977709 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:41.977759 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:41.977843 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:41.979163 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:41.979691 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:41.980892 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:41.981221 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:41.981339 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:41.984736 (+ 3397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:41.984743 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:41.984748 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:41.984780 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:41.986411 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:41.987041 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:41.987044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:41.987045 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:41.987129 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:41.987551 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:41.987554 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:41.987555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:41.987733 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:41.987809 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:41.988500 (+ 691us) server_negotiation.cc:299] Negotiation successful
0619 12:24:41.988625 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 12:24:41.989995 10966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:41.975637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34917 (local address 127.6.231.194:48949)
0619 12:24:41.976122 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:41.976173 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:41.977423 (+ 1250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:41.978053 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:41.978064 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:41.978477 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:41.978987 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:41.979002 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:41.979834 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:41.979837 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:41.980753 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:41.980762 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:41.981504 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:41.981508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:41.981676 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:41.982629 (+ 953us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:41.982645 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:41.984561 (+ 1916us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:41.986535 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:41.986544 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:41.986562 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:41.986932 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:41.987243 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:41.987249 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:41.987253 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:41.987447 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:41.987844 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:41.987851 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:41.988214 (+ 363us) client_negotiation.cc:769] Sending connection context
0619 12:24:41.988456 (+ 242us) client_negotiation.cc:240] Negotiation successful
0619 12:24:41.988840 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":129,"threads_started":1}
I20260619 12:24:41.991228 10964 heartbeater.cc:347] Connected to a master server at 127.6.231.254:34917
I20260619 12:24:41.991489 10964 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
I20260619 12:24:41.992017 10964 heartbeater.cc:511] Master 127.6.231.254:34917 requested a full tablet report, sending...
I20260619 12:24:41.993151 10636 ts_manager.cc:194] Registered new tserver with Master: 30067b4cd3d84a5eb5a0b86b95566fba (127.6.231.194:43567)
I20260619 12:24:41.993839 10636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:48949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:24:42.010748 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:34917
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:46525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:42.125021 10969 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:42.125301 10969 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:42.125344 10969 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:42.129576 10969 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:42.129645 10969 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:42.129704 10969 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:24:42.135092 10969 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:46525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:34917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10969
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:42.136128 10969 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:42.137101 10969 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:42.146570 10977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:42.146687 10974 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:42.146582 10975 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:42.146797 10969 server_base.cc:1034] running on GCE node
I20260619 12:24:42.147313 10969 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:42.147974 10969 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:42.149206 10969 hybrid_clock.cc:648] HybridClock initialized: now 1781871882149187 us; error 33 us; skew 500 ppm
Jun 19 12:24:42 dist-test-slave-kr9h krb5kdc[10594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871882, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:42.152120 10969 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:24:42.152910 10969 webserver.cc:460] Webserver started at http://127.6.231.195:45021/ using document root <none> and password file <none>
I20260619 12:24:42.153414 10969 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:42.153541 10969 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:42.153710 10969 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:42.155766 10969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "6187a0ee253244ca996f6f660832f43f"
format_stamp: "Formatted at 2026-06-19 12:24:42 on dist-test-slave-kr9h"
server_key: "9ebb12225d85b6d949680abada4f6cbe"
server_key_iv: "f2d8f73068e1f0c9d26f8e0874e682cb"
server_key_version: "encryptionkey@0"
I20260619 12:24:42.156277 10969 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "6187a0ee253244ca996f6f660832f43f"
format_stamp: "Formatted at 2026-06-19 12:24:42 on dist-test-slave-kr9h"
server_key: "9ebb12225d85b6d949680abada4f6cbe"
server_key_iv: "f2d8f73068e1f0c9d26f8e0874e682cb"
server_key_version: "encryptionkey@0"
I20260619 12:24:42.160295 10969 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260619 12:24:42.162868 10984 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:42.164215 10969 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:42.164323 10969 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "6187a0ee253244ca996f6f660832f43f"
format_stamp: "Formatted at 2026-06-19 12:24:42 on dist-test-slave-kr9h"
server_key: "9ebb12225d85b6d949680abada4f6cbe"
server_key_iv: "f2d8f73068e1f0c9d26f8e0874e682cb"
server_key_version: "encryptionkey@0"
I20260619 12:24:42.164412 10969 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:42.180778 10969 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:42.184512 10969 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:42.184741 10969 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:42.185381 10969 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:42.186280 10969 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:42.186326 10969 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:42.186398 10969 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:42.186436 10969 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:42.195284 10969 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:43631
I20260619 12:24:42.195343 11097 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:43631 every 8 connection(s)
I20260619 12:24:42.196234 10969 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:24:42.197670 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 10969
I20260619 12:24:42.197798 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:24:42.198137 7071 external_mini_cluster.cc:1428] Setting key b491380877af9cf363422090f0654694
Jun 19 12:24:42 dist-test-slave-kr9h krb5kdc[10594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871882, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:42.210963 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:42.198406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:57371 (local address 127.6.231.254:34917)
0619 12:24:42.198758 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:42.198763 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:42.199595 (+ 832us) server_negotiation.cc:407] Connection header received
0619 12:24:42.201335 (+ 1740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:42.201337 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:42.201382 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:42.201530 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:42.202774 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:42.203296 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:42.204427 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:42.204646 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:42.204730 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:42.207268 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:42.207272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:42.207274 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:42.207298 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:42.208914 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:42.209363 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:42.209366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:42.209367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:42.209502 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:42.209889 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:42.209892 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:42.209893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:42.210079 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:42.210159 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:42.210673 (+ 514us) server_negotiation.cc:299] Negotiation successful
0619 12:24:42.210768 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256}
I20260619 12:24:42.212045 11100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:42.198886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34917 (local address 127.6.231.195:57371)
0619 12:24:42.199406 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:42.199463 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:42.200973 (+ 1510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:42.201726 (+ 753us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:42.201733 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:42.202149 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:42.202635 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:42.202651 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:42.203413 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:42.203416 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:42.204311 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:42.204319 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:42.204789 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:42.204792 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:42.204939 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:42.205656 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:42.205672 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:42.207116 (+ 1444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:42.209020 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:42.209025 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:42.209036 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:42.209276 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:42.209623 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:42.209625 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:42.209627 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:42.209766 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:42.210176 (+ 410us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:42.210181 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:42.210408 (+ 227us) client_negotiation.cc:769] Sending connection context
0619 12:24:42.210622 (+ 214us) client_negotiation.cc:240] Negotiation successful
0619 12:24:42.210945 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":149,"threads_started":1}
I20260619 12:24:42.213178 11098 heartbeater.cc:347] Connected to a master server at 127.6.231.254:34917
I20260619 12:24:42.213479 11098 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:42.213966 11098 heartbeater.cc:511] Master 127.6.231.254:34917 requested a full tablet report, sending...
I20260619 12:24:42.215032 10636 ts_manager.cc:194] Registered new tserver with Master: 6187a0ee253244ca996f6f660832f43f (127.6.231.195:43631)
I20260619 12:24:42.215632 10636 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:57371
I20260619 12:24:42.224207 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:24:42 dist-test-slave-kr9h krb5kdc[10594](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:24:42 dist-test-slave-kr9h krb5kdc[10594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
I20260619 12:24:42.251641 11105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:42.241768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53716 (local address 127.6.231.193:46463)
0619 12:24:42.242166 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:42.242181 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:42.242206 (+ 25us) server_negotiation.cc:407] Connection header received
0619 12:24:42.242256 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:42.242261 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:42.242450 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:42.242601 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:42.243337 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:42.244137 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:42.244981 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:42.245381 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:42.245483 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:42.247943 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:42.247950 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:42.247963 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:42.247985 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:42.249764 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:42.250164 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:42.250169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:42.250171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:42.250251 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:42.250504 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:42.250507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:42.250509 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:42.250782 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:42.250962 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:42.251243 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 12:24:42.251421 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":137,"threads_started":1}
Jun 19 12:24:42 dist-test-slave-kr9h krb5kdc[10594](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871882, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:42.263654 10678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:42.254870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40328 (local address 127.6.231.254:34917)
0619 12:24:42.255034 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:42.255039 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:42.255142 (+ 103us) server_negotiation.cc:407] Connection header received
0619 12:24:42.255302 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:42.255305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:42.255358 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:42.255445 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:42.256047 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:42.256615 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:42.257418 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:42.257675 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:42.257794 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:42.260277 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:42.260281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:42.260284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:42.260314 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:42.261853 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:42.262316 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:42.262322 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:42.262324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:42.262405 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:42.262702 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:42.262705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:42.262707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:42.262934 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:42.263092 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:42.263336 (+ 244us) server_negotiation.cc:299] Negotiation successful
0619 12:24:42.263475 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260619 12:24:42.264132 10636 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40328
I20260619 12:24:42.264777 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 10701
I20260619 12:24:42.270776 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 10835
I20260619 12:24:42.276043 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 10969
I20260619 12:24:42.281319 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 10607
2026-06-19T12:24:42Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2191 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:24:42 dist-test-slave-kr9h krb5kdc[11110](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:24:42 dist-test-slave-kr9h krb5kdc[11110](info): set up 2 sockets
Jun 19 12:24:42 dist-test-slave-kr9h krb5kdc[11110](info): commencing operation
krb5kdc: starting...
W20260619 12:24:43.345573 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.005s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:24:43 dist-test-slave-kr9h krb5kdc[11110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:24:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:24:43.474004 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:43355
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:43355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:43.592459 11123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:43.592736 11123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:43.592777 11123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:43.596889 11123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:43.596961 11123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:43.596987 11123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:43.597004 11123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:43.597020 11123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:43.602497 11123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:43355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:43355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:43.603595 11123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:43.604642 11123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:43.613212 11128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:43.613235 11131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:43.613235 11129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:43.613693 11123 server_base.cc:1034] running on GCE node
I20260619 12:24:43.614415 11123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:43.615629 11123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:43.616858 11123 hybrid_clock.cc:648] HybridClock initialized: now 1781871883616799 us; error 81 us; skew 500 ppm
Jun 19 12:24:43 dist-test-slave-kr9h krb5kdc[11110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871883, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:43.620114 11123 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:24:43.620857 11123 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:43.620927 11123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:43.621106 11123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:43.623201 11123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "45ccc1140cef4b5f9410394c597cfe3b"
format_stamp: "Formatted at 2026-06-19 12:24:43 on dist-test-slave-kr9h"
server_key: "215d22c5d3926559ae48021937212d34"
server_key_iv: "56f03d1080319dbbd9970da520239ce8"
server_key_version: "encryptionkey@0"
I20260619 12:24:43.623718 11123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "45ccc1140cef4b5f9410394c597cfe3b"
format_stamp: "Formatted at 2026-06-19 12:24:43 on dist-test-slave-kr9h"
server_key: "215d22c5d3926559ae48021937212d34"
server_key_iv: "56f03d1080319dbbd9970da520239ce8"
server_key_version: "encryptionkey@0"
I20260619 12:24:43.627525 11123 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.005s
I20260619 12:24:43.629948 11137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:43.631233 11123 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.002s
I20260619 12:24:43.631335 11123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "45ccc1140cef4b5f9410394c597cfe3b"
format_stamp: "Formatted at 2026-06-19 12:24:43 on dist-test-slave-kr9h"
server_key: "215d22c5d3926559ae48021937212d34"
server_key_iv: "56f03d1080319dbbd9970da520239ce8"
server_key_version: "encryptionkey@0"
I20260619 12:24:43.631423 11123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:43.646984 11123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:43.650225 11123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:43.650377 11123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:43.657186 11123 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:43355
I20260619 12:24:43.657542 11189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:43355 every 8 connection(s)
I20260619 12:24:43.657998 11123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:43.660905 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 11123
I20260619 12:24:43.661127 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:43.661139 11190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:43.661536 7071 external_mini_cluster.cc:1428] Setting key 0b7708eff9b84f73846228331d0b071e
I20260619 12:24:43.668352 11190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b: Bootstrap starting.
I20260619 12:24:43.671042 11190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b: Neither blocks nor log segments found. Creating new log.
Jun 19 12:24:43 dist-test-slave-kr9h krb5kdc[11110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871883, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:43.672115 11190 log.cc:824] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:43.674715 11190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b: No bootstrap required, opened a new log
I20260619 12:24:43.677380 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:43.663064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45254 (local address 127.6.231.254:43355)
0619 12:24:43.663561 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:43.663577 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:43.663615 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:24:43.664614 (+ 999us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:43.664635 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:43.664987 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:43.665354 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:43.666201 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:43.667218 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:43.668154 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:43.668763 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:43.668956 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:43.671968 (+ 3012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:43.671975 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:43.671993 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:43.672023 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:43.674276 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:43.674876 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:43.674882 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:43.674887 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:43.675016 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:43.675340 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:43.675343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:43.675344 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:43.675781 (+ 437us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:43.675934 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:43.676260 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 12:24:43.676596 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":143,"threads_started":1}
I20260619 12:24:43.678918 11190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ccc1140cef4b5f9410394c597cfe3b" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43355 } }
I20260619 12:24:43.679162 11190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:43.679225 11190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45ccc1140cef4b5f9410394c597cfe3b, State: Initialized, Role: FOLLOWER
I20260619 12:24:43.679638 11190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ccc1140cef4b5f9410394c597cfe3b" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43355 } }
I20260619 12:24:43.679756 11190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:43.679903 11190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:43.680017 11190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:43.681006 11190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ccc1140cef4b5f9410394c597cfe3b" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43355 } }
I20260619 12:24:43.681365 11190 leader_election.cc:302] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 45ccc1140cef4b5f9410394c597cfe3b; no voters:
I20260619 12:24:43.681768 11190 leader_election.cc:288] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:43.681890 11195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:43.682142 11195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [term 1 LEADER]: Becoming Leader. State: Replica: 45ccc1140cef4b5f9410394c597cfe3b, State: Running, Role: LEADER
I20260619 12:24:43.682485 11195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ccc1140cef4b5f9410394c597cfe3b" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43355 } }
I20260619 12:24:43.683058 11190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:43.684391 11196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "45ccc1140cef4b5f9410394c597cfe3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ccc1140cef4b5f9410394c597cfe3b" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43355 } } }
I20260619 12:24:43.684505 11196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:43.684438 11197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 45ccc1140cef4b5f9410394c597cfe3b. Latest consensus state: current_term: 1 leader_uuid: "45ccc1140cef4b5f9410394c597cfe3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "45ccc1140cef4b5f9410394c597cfe3b" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43355 } } }
I20260619 12:24:43.684557 11197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:43.684943 11204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:43.688215 11204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:43.694023 11204 catalog_manager.cc:1347] Generated new cluster ID: 7f35afaddd5d4b5fbc76d54a49bd0cd9
I20260619 12:24:43.694131 11204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:43.700345 11204 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:43.701326 11204 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:43.705551 11204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b: Generated new TSK 0
I20260619 12:24:43.706245 11204 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:24:43.745888 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:43355
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:43.872426 11216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:43.872684 11216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:43.872728 11216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:43.876765 11216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:43.876838 11216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:43.876900 11216 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:24:43.882586 11216 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:43355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:43.883666 11216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:43.884791 11216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:43.894878 11224 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:43.894879 11221 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:43.894883 11222 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:43.895466 11216 server_base.cc:1034] running on GCE node
I20260619 12:24:43.895845 11216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:43.896545 11216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:43.897738 11216 hybrid_clock.cc:648] HybridClock initialized: now 1781871883897656 us; error 98 us; skew 500 ppm
Jun 19 12:24:43 dist-test-slave-kr9h krb5kdc[11110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871883, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:43.900682 11216 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:24:43.901520 11216 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:43.901609 11216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:43.901827 11216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:43.903954 11216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "a0c0305a936046b6b6ced8b2d8b51e30"
format_stamp: "Formatted at 2026-06-19 12:24:43 on dist-test-slave-kr9h"
server_key: "8f6eec8c16f81675d18b4948ea242c93"
server_key_iv: "cf7d8f81eda3e419b355547cc0d21e3f"
server_key_version: "encryptionkey@0"
I20260619 12:24:43.904456 11216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "a0c0305a936046b6b6ced8b2d8b51e30"
format_stamp: "Formatted at 2026-06-19 12:24:43 on dist-test-slave-kr9h"
server_key: "8f6eec8c16f81675d18b4948ea242c93"
server_key_iv: "cf7d8f81eda3e419b355547cc0d21e3f"
server_key_version: "encryptionkey@0"
I20260619 12:24:43.908170 11216 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 12:24:43.910616 11230 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:43.911968 11216 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:43.912127 11216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "a0c0305a936046b6b6ced8b2d8b51e30"
format_stamp: "Formatted at 2026-06-19 12:24:43 on dist-test-slave-kr9h"
server_key: "8f6eec8c16f81675d18b4948ea242c93"
server_key_iv: "cf7d8f81eda3e419b355547cc0d21e3f"
server_key_version: "encryptionkey@0"
I20260619 12:24:43.912247 11216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:43.954411 11216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:43.958007 11216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:43.958231 11216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:43.958931 11216 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:43.959780 11216 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:43.959858 11216 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:43.959955 11216 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:43.960006 11216 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:43.969379 11216 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:45257
I20260619 12:24:43.969401 11343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:45257 every 8 connection(s)
I20260619 12:24:43.970039 11216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:24:43.973831 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 11216
I20260619 12:24:43.974009 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:24:43.974339 7071 external_mini_cluster.cc:1428] Setting key a544c6a63cd23c5ffba16362c00e06b9
Jun 19 12:24:43 dist-test-slave-kr9h krb5kdc[11110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871883, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
I20260619 12:24:43.985546 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:43.971885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:42375 (local address 127.6.231.254:43355)
0619 12:24:43.972053 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:43.972057 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:43.973017 (+ 960us) server_negotiation.cc:407] Connection header received
0619 12:24:43.974496 (+ 1479us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:43.974499 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:43.974557 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:43.974634 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:43.975795 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:43.976338 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:43.977568 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:43.977820 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:43.977954 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:43.981211 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:43.981214 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:43.981216 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:43.981241 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:43.983058 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:43.983777 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:43.983780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:43.983781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:43.983867 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:43.984298 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:43.984301 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:43.984302 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:43.984496 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:43.984578 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:43.985184 (+ 606us) server_negotiation.cc:299] Negotiation successful
0619 12:24:43.985328 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 12:24:43.986415 11346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:43.972321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:43355 (local address 127.6.231.193:42375)
0619 12:24:43.972762 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:43.972805 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:43.974250 (+ 1445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:43.974803 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:43.974810 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:43.975162 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:43.975644 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:43.975657 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:43.976476 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:43.976480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:43.977366 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:43.977374 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:43.977993 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:43.977998 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:43.978242 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:43.978952 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:43.978978 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:43.980997 (+ 2019us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:43.983186 (+ 2189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:43.983195 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:43.983214 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:43.983622 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:43.983985 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:43.983991 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:43.983996 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:43.984200 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:43.984600 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:43.984605 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:43.984918 (+ 313us) client_negotiation.cc:769] Sending connection context
0619 12:24:43.985133 (+ 215us) client_negotiation.cc:240] Negotiation successful
0619 12:24:43.985476 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":105,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:24:43.987871 11344 heartbeater.cc:347] Connected to a master server at 127.6.231.254:43355
I20260619 12:24:43.988147 11344 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:43.988741 11344 heartbeater.cc:511] Master 127.6.231.254:43355 requested a full tablet report, sending...
I20260619 12:24:43.990527 11154 ts_manager.cc:194] Registered new tserver with Master: a0c0305a936046b6b6ced8b2d8b51e30 (127.6.231.193:45257)
I20260619 12:24:43.992115 11154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:42375
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:24:44.003023 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:43355
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:44.115787 11349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:44.116062 11349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:44.116168 11349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:44.120486 11349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:44.120555 11349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:44.120617 11349 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:24:44.126120 11349 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:43355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:44.127260 11349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:44.128314 11349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:44.138254 11354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:44.138213 11357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:44.138221 11355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:44.138466 11349 server_base.cc:1034] running on GCE node
I20260619 12:24:44.139045 11349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:44.139707 11349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:44.140877 11349 hybrid_clock.cc:648] HybridClock initialized: now 1781871884140861 us; error 28 us; skew 500 ppm
Jun 19 12:24:44 dist-test-slave-kr9h krb5kdc[11110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871884, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:44.143790 11349 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:24:44.144541 11349 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:44.144604 11349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:44.144768 11349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:44.146875 11349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "0da017ab35144c628450472df2a990f7"
format_stamp: "Formatted at 2026-06-19 12:24:44 on dist-test-slave-kr9h"
server_key: "5aba81801109a488afe49c86bbaaa819"
server_key_iv: "dec7731d335467370560f61f6cb707b0"
server_key_version: "encryptionkey@0"
I20260619 12:24:44.147377 11349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "0da017ab35144c628450472df2a990f7"
format_stamp: "Formatted at 2026-06-19 12:24:44 on dist-test-slave-kr9h"
server_key: "5aba81801109a488afe49c86bbaaa819"
server_key_iv: "dec7731d335467370560f61f6cb707b0"
server_key_version: "encryptionkey@0"
I20260619 12:24:44.151149 11349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:24:44.153591 11363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:44.154841 11349 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:24:44.154979 11349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "0da017ab35144c628450472df2a990f7"
format_stamp: "Formatted at 2026-06-19 12:24:44 on dist-test-slave-kr9h"
server_key: "5aba81801109a488afe49c86bbaaa819"
server_key_iv: "dec7731d335467370560f61f6cb707b0"
server_key_version: "encryptionkey@0"
I20260619 12:24:44.155069 11349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:44.187520 11349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:44.190843 11349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:44.191006 11349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:44.191540 11349 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:44.192477 11349 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:44.192528 11349 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:44.192574 11349 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:44.192591 11349 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:44.202409 11349 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:38115
I20260619 12:24:44.202423 11476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:38115 every 8 connection(s)
I20260619 12:24:44.203037 11349 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:24:44.209978 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 11349
I20260619 12:24:44.210104 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:24:44.210377 7071 external_mini_cluster.cc:1428] Setting key 7090abaa3b238ea285ceb6ac91808233
Jun 19 12:24:44 dist-test-slave-kr9h krb5kdc[11110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871884, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:44.218432 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.204937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:46363 (local address 127.6.231.254:43355)
0619 12:24:44.205111 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:44.205115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:44.206359 (+ 1244us) server_negotiation.cc:407] Connection header received
0619 12:24:44.207395 (+ 1036us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:44.207398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:44.207453 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:44.207533 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:44.208744 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.209298 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.210734 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.210968 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.211096 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.213686 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:44.213693 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:44.213697 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:44.213728 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:44.215548 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:44.216244 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:44.216248 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:44.216250 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:44.216343 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:44.217022 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:44.217036 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:44.217039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:44.217321 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:44.217474 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:44.218107 (+ 633us) server_negotiation.cc:299] Negotiation successful
0619 12:24:44.218248 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 12:24:44.219254 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.205345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:43355 (local address 127.6.231.194:46363)
0619 12:24:44.206201 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:44.206242 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:44.207185 (+ 943us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:44.207707 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:44.207713 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:44.208114 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:44.208595 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.208608 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.209458 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.209462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.210614 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.210623 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.211133 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.211138 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.211306 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.211946 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:44.211962 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:44.213516 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:44.215685 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:44.215690 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:44.215703 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:44.216113 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:44.216705 (+ 592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:44.216710 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:44.216712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:44.216871 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:44.217537 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:44.217543 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:44.217799 (+ 256us) client_negotiation.cc:769] Sending connection context
0619 12:24:44.217987 (+ 188us) client_negotiation.cc:240] Negotiation successful
0619 12:24:44.218264 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":128,"threads_started":1}
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
I20260619 12:24:44.220508 11477 heartbeater.cc:347] Connected to a master server at 127.6.231.254:43355
I20260619 12:24:44.220732 11477 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:44.221268 11477 heartbeater.cc:511] Master 127.6.231.254:43355 requested a full tablet report, sending...
I20260619 12:24:44.222512 11154 ts_manager.cc:194] Registered new tserver with Master: 0da017ab35144c628450472df2a990f7 (127.6.231.194:38115)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
I20260619 12:24:44.223281 11154 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:46363
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:24:44.240255 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:43355
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44829
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:44.350714 11482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:44.350984 11482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:44.351029 11482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:44.354951 11482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:44.355011 11482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:44.355072 11482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:24:44.360545 11482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44829
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:43355
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:44.361586 11482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:44.362593 11482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:44.372085 11487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:44.372227 11490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:44.372328 11488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:44.372599 11482 server_base.cc:1034] running on GCE node
I20260619 12:24:44.373046 11482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:44.373853 11482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:44.375046 11482 hybrid_clock.cc:648] HybridClock initialized: now 1781871884375020 us; error 38 us; skew 500 ppm
Jun 19 12:24:44 dist-test-slave-kr9h krb5kdc[11110](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871884, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:44.378008 11482 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:24:44.378728 11482 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:44.378826 11482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:44.379063 11482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:44.381183 11482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "51840727ccdd48b19491c84fd65feeab"
format_stamp: "Formatted at 2026-06-19 12:24:44 on dist-test-slave-kr9h"
server_key: "8c5b073dc64f7065f8255c00eb07903d"
server_key_iv: "b9979957f785e4c6346d70ec7b483230"
server_key_version: "encryptionkey@0"
I20260619 12:24:44.381783 11482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "51840727ccdd48b19491c84fd65feeab"
format_stamp: "Formatted at 2026-06-19 12:24:44 on dist-test-slave-kr9h"
server_key: "8c5b073dc64f7065f8255c00eb07903d"
server_key_iv: "b9979957f785e4c6346d70ec7b483230"
server_key_version: "encryptionkey@0"
I20260619 12:24:44.385753 11482 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 12:24:44.388109 11496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:44.389343 11482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:24:44.389523 11482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "51840727ccdd48b19491c84fd65feeab"
format_stamp: "Formatted at 2026-06-19 12:24:44 on dist-test-slave-kr9h"
server_key: "8c5b073dc64f7065f8255c00eb07903d"
server_key_iv: "b9979957f785e4c6346d70ec7b483230"
server_key_version: "encryptionkey@0"
I20260619 12:24:44.389644 11482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:44.399142 11482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:44.402488 11482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:44.402686 11482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:44.403246 11482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:44.404093 11482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:44.404137 11482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:44.404177 11482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:44.404192 11482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:44.413527 11482 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:37935
I20260619 12:24:44.413556 11609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:37935 every 8 connection(s)
I20260619 12:24:44.414222 11482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:24:44.416605 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 11482
I20260619 12:24:44.416720 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:24:44.417069 7071 external_mini_cluster.cc:1428] Setting key a6712d17ec655a4fd20f762ac12dba17
Jun 19 12:24:44 dist-test-slave-kr9h krb5kdc[11110](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871884, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:44.429001 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.416182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:58201 (local address 127.6.231.254:43355)
0619 12:24:44.416322 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:44.416326 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:44.417962 (+ 1636us) server_negotiation.cc:407] Connection header received
0619 12:24:44.419082 (+ 1120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:44.419085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:44.419136 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:44.419236 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:44.420601 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.421190 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.422497 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.422715 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.422801 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.425253 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:44.425255 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:44.425257 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:44.425281 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:44.426901 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:44.427390 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:44.427393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:44.427395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:44.427490 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:44.427813 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:44.427815 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:44.427817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:44.428081 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:44.428178 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:44.428694 (+ 516us) server_negotiation.cc:299] Negotiation successful
0619 12:24:44.428807 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:24:44.429920 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.416647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:43355 (local address 127.6.231.195:58201)
0619 12:24:44.417729 (+ 1082us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:44.417787 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:44.418854 (+ 1067us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:44.419468 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:44.419480 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:44.419943 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:44.420440 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.420454 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.421304 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.421307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.422348 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.422357 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.422852 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.422855 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.423014 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.423558 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:44.423574 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:44.425042 (+ 1468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:44.427021 (+ 1979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:44.427026 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:44.427040 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:44.427287 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:44.427585 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:44.427587 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:44.427589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:44.427721 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:44.428216 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:44.428221 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:44.428462 (+ 241us) client_negotiation.cc:769] Sending connection context
0619 12:24:44.428628 (+ 166us) client_negotiation.cc:240] Negotiation successful
0619 12:24:44.428881 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":133,"threads_started":1}
I20260619 12:24:44.431380 11610 heartbeater.cc:347] Connected to a master server at 127.6.231.254:43355
I20260619 12:24:44.431674 11610 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:44.432128 11610 heartbeater.cc:511] Master 127.6.231.254:43355 requested a full tablet report, sending...
I20260619 12:24:44.433127 11151 ts_manager.cc:194] Registered new tserver with Master: 51840727ccdd48b19491c84fd65feeab (127.6.231.195:37935)
I20260619 12:24:44.433764 11151 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:58201
I20260619 12:24:44.443899 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:24:44.453353 11193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.445948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45268 (local address 127.6.231.254:43355)
0619 12:24:44.446098 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:44.446101 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:44.446274 (+ 173us) server_negotiation.cc:407] Connection header received
0619 12:24:44.446372 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:44.446374 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:44.446417 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:44.446487 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:44.447174 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.447690 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.448527 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.448736 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.448815 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.450272 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:44.450275 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:44.450277 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:44.450299 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:44.451847 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:44.452229 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:44.452231 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:44.452233 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:44.452303 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:44.452600 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:44.452602 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:44.452603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:44.452815 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:44.452959 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:44.453143 (+ 184us) server_negotiation.cc:299] Negotiation successful
0619 12:24:44.453234 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 12:24:44.456513 11151 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:24:44.459676 11151 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:44.474143 11625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.468677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50992 (local address 127.6.231.193:45257)
0619 12:24:44.469368 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:44.469377 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:44.469397 (+ 20us) server_negotiation.cc:407] Connection header received
0619 12:24:44.469491 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:44.469497 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:44.469686 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:44.469831 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:44.470272 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.471035 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.472740 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.473358 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.473543 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.473695 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:44.473854 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 12:24:44.473969 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":87,"threads_started":1}
I20260619 12:24:44.474162 11624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.467957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:37935 (local address 127.0.0.1:60578)
0619 12:24:44.468825 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:44.468875 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:44.469016 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:44.469829 (+ 813us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:44.469834 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:44.469855 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:44.470023 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.470030 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.471525 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.471529 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.472550 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.472559 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.473651 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.473654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.473765 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.473785 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 12:24:44.473832 (+ 47us) client_negotiation.cc:240] Negotiation successful
0619 12:24:44.473988 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"mutex_wait_us":5,"spinlock_wait_cycles":6784,"thread_start_us":70,"threads_started":1}
I20260619 12:24:44.474139 11622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.468149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:45257 (local address 127.0.0.1:50992)
0619 12:24:44.468825 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:44.468877 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:44.469015 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:44.470001 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:44.470003 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:44.470030 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:44.470149 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.470154 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.471184 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.471187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.472600 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.472609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.473561 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.473565 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.473666 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.473702 (+ 36us) client_negotiation.cc:769] Sending connection context
0619 12:24:44.473815 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 12:24:44.474001 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"mutex_wait_us":159,"spinlock_wait_cycles":22912,"thread_start_us":107,"threads_started":1}
I20260619 12:24:44.474644 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.468142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60578 (local address 127.6.231.195:37935)
0619 12:24:44.469293 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:44.469310 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:44.469332 (+ 22us) server_negotiation.cc:407] Connection header received
0619 12:24:44.469407 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:44.469414 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:44.469664 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:44.469832 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:44.470155 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.471387 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.472678 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.473509 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.474224 (+ 715us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.474368 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:44.474413 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 12:24:44.474498 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1054,"thread_start_us":135,"threads_started":1}
I20260619 12:24:44.474735 11621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.468349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:38115 (local address 127.0.0.1:40254)
0619 12:24:44.469224 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:44.469237 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:44.469312 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:44.469991 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:44.469994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:44.470028 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:44.470168 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.470175 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.471599 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.471602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.472751 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.472758 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.473957 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.473962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.474039 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.474061 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 12:24:44.474128 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 12:24:44.474193 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"mutex_wait_us":4,"spinlock_wait_cycles":3968,"thread_start_us":100,"threads_started":1}
I20260619 12:24:44.475208 11626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.468754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40254 (local address 127.6.231.194:38115)
0619 12:24:44.469182 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:44.469194 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:44.469400 (+ 206us) server_negotiation.cc:407] Connection header received
0619 12:24:44.469504 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:44.469509 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:44.469686 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:44.469831 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:44.470301 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.471401 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.472858 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.473662 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.473754 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.473886 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:44.474984 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0619 12:24:44.475072 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":111,"threads_started":1}
I20260619 12:24:44.477357 11411 tablet_service.cc:1467] Processing CreateTablet for tablet 6f82c842fff3419097c68ad733df402d (DEFAULT_TABLE table=test-table [id=7e5b47e761ba4cd5bb2676403ee80374]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:44.477386 11544 tablet_service.cc:1467] Processing CreateTablet for tablet 6f82c842fff3419097c68ad733df402d (DEFAULT_TABLE table=test-table [id=7e5b47e761ba4cd5bb2676403ee80374]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:44.477386 11278 tablet_service.cc:1467] Processing CreateTablet for tablet 6f82c842fff3419097c68ad733df402d (DEFAULT_TABLE table=test-table [id=7e5b47e761ba4cd5bb2676403ee80374]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:44.478318 11411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f82c842fff3419097c68ad733df402d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:44.478371 11278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f82c842fff3419097c68ad733df402d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:44.478376 11544 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f82c842fff3419097c68ad733df402d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:44.484548 11627 tablet_bootstrap.cc:492] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7: Bootstrap starting.
I20260619 12:24:44.485882 11628 tablet_bootstrap.cc:492] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab: Bootstrap starting.
I20260619 12:24:44.486474 11629 tablet_bootstrap.cc:492] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30: Bootstrap starting.
I20260619 12:24:44.487056 11627 tablet_bootstrap.cc:654] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:44.488010 11627 log.cc:824] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:44.488312 11629 tablet_bootstrap.cc:654] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:44.489077 11629 log.cc:824] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:44.490172 11627 tablet_bootstrap.cc:492] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7: No bootstrap required, opened a new log
I20260619 12:24:44.490317 11628 tablet_bootstrap.cc:654] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:44.490373 11627 ts_tablet_manager.cc:1397] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260619 12:24:44.490761 11629 tablet_bootstrap.cc:492] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30: No bootstrap required, opened a new log
I20260619 12:24:44.490965 11629 ts_tablet_manager.cc:1397] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 12:24:44.491394 11628 log.cc:824] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:44.493346 11628 tablet_bootstrap.cc:492] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab: No bootstrap required, opened a new log
I20260619 12:24:44.493549 11628 ts_tablet_manager.cc:1397] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260619 12:24:44.494329 11627 raft_consensus.cc:348] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51840727ccdd48b19491c84fd65feeab" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37935 } } peers { permanent_uuid: "a0c0305a936046b6b6ced8b2d8b51e30" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45257 } } peers { permanent_uuid: "0da017ab35144c628450472df2a990f7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 38115 } }
I20260619 12:24:44.494585 11627 raft_consensus.cc:374] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:44.494637 11627 raft_consensus.cc:729] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da017ab35144c628450472df2a990f7, State: Initialized, Role: FOLLOWER
I20260619 12:24:44.495065 11627 consensus_queue.cc:260] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51840727ccdd48b19491c84fd65feeab" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37935 } } peers { permanent_uuid: "a0c0305a936046b6b6ced8b2d8b51e30" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45257 } } peers { permanent_uuid: "0da017ab35144c628450472df2a990f7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 38115 } }
I20260619 12:24:44.495289 11629 raft_consensus.cc:348] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51840727ccdd48b19491c84fd65feeab" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37935 } } peers { permanent_uuid: "a0c0305a936046b6b6ced8b2d8b51e30" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45257 } } peers { permanent_uuid: "0da017ab35144c628450472df2a990f7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 38115 } }
I20260619 12:24:44.495527 11629 raft_consensus.cc:374] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:44.495581 11629 raft_consensus.cc:729] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0c0305a936046b6b6ced8b2d8b51e30, State: Initialized, Role: FOLLOWER
I20260619 12:24:44.495813 11477 heartbeater.cc:503] Master 127.6.231.254:43355 was elected leader, sending a full tablet report...
I20260619 12:24:44.496094 11627 ts_tablet_manager.cc:1428] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 12:24:44.496057 11629 consensus_queue.cc:260] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51840727ccdd48b19491c84fd65feeab" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37935 } } peers { permanent_uuid: "a0c0305a936046b6b6ced8b2d8b51e30" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45257 } } peers { permanent_uuid: "0da017ab35144c628450472df2a990f7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 38115 } }
I20260619 12:24:44.496702 11344 heartbeater.cc:503] Master 127.6.231.254:43355 was elected leader, sending a full tablet report...
I20260619 12:24:44.496996 11629 ts_tablet_manager.cc:1428] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 12:24:44.497284 11628 raft_consensus.cc:348] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51840727ccdd48b19491c84fd65feeab" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37935 } } peers { permanent_uuid: "a0c0305a936046b6b6ced8b2d8b51e30" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45257 } } peers { permanent_uuid: "0da017ab35144c628450472df2a990f7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 38115 } }
I20260619 12:24:44.497566 11628 raft_consensus.cc:374] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:44.497629 11628 raft_consensus.cc:729] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51840727ccdd48b19491c84fd65feeab, State: Initialized, Role: FOLLOWER
I20260619 12:24:44.498008 11628 consensus_queue.cc:260] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51840727ccdd48b19491c84fd65feeab" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37935 } } peers { permanent_uuid: "a0c0305a936046b6b6ced8b2d8b51e30" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45257 } } peers { permanent_uuid: "0da017ab35144c628450472df2a990f7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 38115 } }
I20260619 12:24:44.498726 11610 heartbeater.cc:503] Master 127.6.231.254:43355 was elected leader, sending a full tablet report...
I20260619 12:24:44.499430 11628 ts_tablet_manager.cc:1428] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 12:24:44.555848 11635 raft_consensus.cc:482] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:44.556043 11635 raft_consensus.cc:504] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51840727ccdd48b19491c84fd65feeab" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37935 } } peers { permanent_uuid: "a0c0305a936046b6b6ced8b2d8b51e30" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45257 } } peers { permanent_uuid: "0da017ab35144c628450472df2a990f7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 38115 } }
I20260619 12:24:44.557171 11635 leader_election.cc:288] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a0c0305a936046b6b6ced8b2d8b51e30 (127.6.231.193:45257), 0da017ab35144c628450472df2a990f7 (127.6.231.194:38115)
I20260619 12:24:44.561401 11626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.557726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:58751 (local address 127.6.231.194:38115)
0619 12:24:44.557955 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:44.557962 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:44.557978 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:24:44.558106 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:44.558109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:44.558186 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:44.558270 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:44.558550 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.559171 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.560385 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.560790 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.560923 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.560976 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:44.561210 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 12:24:44.561278 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 12:24:44.561596 11625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.557515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:44845 (local address 127.6.231.193:45257)
0619 12:24:44.557646 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:44.557651 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:44.557677 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:24:44.557735 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:44.557739 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:44.557786 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:44.557873 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:44.558257 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.558923 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.560402 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.560870 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.561029 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.561092 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:44.561350 (+ 258us) server_negotiation.cc:299] Negotiation successful
0619 12:24:44.561425 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:24:44.561640 11612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.557326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:45257 (local address 127.6.231.195:44845)
0619 12:24:44.557520 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:44.557540 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:44.557630 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:44.557932 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:44.557936 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:44.557983 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:44.558122 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.558129 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.559051 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.559055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.560221 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.560266 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.561014 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.561018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.561195 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.561213 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 12:24:44.561320 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 12:24:44.561410 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":27,"spinlock_wait_cycles":26112}
I20260619 12:24:44.561431 11636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.557521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:38115 (local address 127.6.231.195:58751)
0619 12:24:44.557877 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:44.557890 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:44.557992 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:44.558287 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:44.558289 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:44.558306 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:44.558412 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.558418 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.559331 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.559334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.560280 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:44.560290 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.560911 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.560915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:44.561017 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.561104 (+ 87us) client_negotiation.cc:769] Sending connection context
0619 12:24:44.561195 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 12:24:44.561268 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":131,"spinlock_wait_cycles":6144,"thread_start_us":153,"threads_started":1}
I20260619 12:24:44.562352 11298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f82c842fff3419097c68ad733df402d" candidate_uuid: "51840727ccdd48b19491c84fd65feeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0c0305a936046b6b6ced8b2d8b51e30" is_pre_election: true
I20260619 12:24:44.562508 11431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f82c842fff3419097c68ad733df402d" candidate_uuid: "51840727ccdd48b19491c84fd65feeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da017ab35144c628450472df2a990f7" is_pre_election: true
I20260619 12:24:44.562697 11298 raft_consensus.cc:2454] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51840727ccdd48b19491c84fd65feeab in term 0.
I20260619 12:24:44.562737 11431 raft_consensus.cc:2454] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51840727ccdd48b19491c84fd65feeab in term 0.
I20260619 12:24:44.563215 11498 leader_election.cc:302] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da017ab35144c628450472df2a990f7, 51840727ccdd48b19491c84fd65feeab; no voters:
I20260619 12:24:44.563495 11635 raft_consensus.cc:2781] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:44.563575 11635 raft_consensus.cc:482] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:44.563618 11635 raft_consensus.cc:3037] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:44.564618 11635 raft_consensus.cc:504] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51840727ccdd48b19491c84fd65feeab" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37935 } } peers { permanent_uuid: "a0c0305a936046b6b6ced8b2d8b51e30" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45257 } } peers { permanent_uuid: "0da017ab35144c628450472df2a990f7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 38115 } }
I20260619 12:24:44.564941 11635 leader_election.cc:288] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [CANDIDATE]: Term 1 election: Requested vote from peers a0c0305a936046b6b6ced8b2d8b51e30 (127.6.231.193:45257), 0da017ab35144c628450472df2a990f7 (127.6.231.194:38115)
I20260619 12:24:44.565274 11431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f82c842fff3419097c68ad733df402d" candidate_uuid: "51840727ccdd48b19491c84fd65feeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0da017ab35144c628450472df2a990f7"
I20260619 12:24:44.565331 11298 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f82c842fff3419097c68ad733df402d" candidate_uuid: "51840727ccdd48b19491c84fd65feeab" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a0c0305a936046b6b6ced8b2d8b51e30"
I20260619 12:24:44.565431 11431 raft_consensus.cc:3037] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:44.565503 11298 raft_consensus.cc:3037] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:44.566499 11431 raft_consensus.cc:2454] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51840727ccdd48b19491c84fd65feeab in term 1.
I20260619 12:24:44.566602 11298 raft_consensus.cc:2454] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51840727ccdd48b19491c84fd65feeab in term 1.
I20260619 12:24:44.566838 11498 leader_election.cc:302] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0da017ab35144c628450472df2a990f7, 51840727ccdd48b19491c84fd65feeab; no voters:
I20260619 12:24:44.567019 11635 raft_consensus.cc:2781] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:44.567235 11635 raft_consensus.cc:686] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [term 1 LEADER]: Becoming Leader. State: Replica: 51840727ccdd48b19491c84fd65feeab, State: Running, Role: LEADER
I20260619 12:24:44.567610 11635 consensus_queue.cc:237] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51840727ccdd48b19491c84fd65feeab" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37935 } } peers { permanent_uuid: "a0c0305a936046b6b6ced8b2d8b51e30" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45257 } } peers { permanent_uuid: "0da017ab35144c628450472df2a990f7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 38115 } }
I20260619 12:24:44.570883 11154 catalog_manager.cc:5495] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab reported cstate change: term changed from 0 to 1, leader changed from <none> to 51840727ccdd48b19491c84fd65feeab (127.6.231.195). New cstate: current_term: 1 leader_uuid: "51840727ccdd48b19491c84fd65feeab" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "51840727ccdd48b19491c84fd65feeab" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37935 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a0c0305a936046b6b6ced8b2d8b51e30" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0da017ab35144c628450472df2a990f7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 38115 } health_report { overall_health: UNKNOWN } } }
I20260619 12:24:44.583408 11623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.579575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60582 (local address 127.6.231.195:37935)
0619 12:24:44.579707 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:44.579713 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:44.579771 (+ 58us) server_negotiation.cc:407] Connection header received
0619 12:24:44.579950 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:44.579952 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:44.580028 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:44.580150 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:44.580403 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.580980 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.581979 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:44.582207 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:44.582323 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:44.582624 (+ 301us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:44.583031 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:44.583170 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:44.583210 (+ 40us) server_negotiation.cc:299] Negotiation successful
0619 12:24:44.583263 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:24:44.589596 11431 raft_consensus.cc:1261] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7 [term 1 FOLLOWER]: Refusing update from remote peer 51840727ccdd48b19491c84fd65feeab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:44.589596 11298 raft_consensus.cc:1261] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30 [term 1 FOLLOWER]: Refusing update from remote peer 51840727ccdd48b19491c84fd65feeab: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:44.590464 11635 consensus_queue.cc:1035] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "0da017ab35144c628450472df2a990f7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 38115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:44.590691 11637 consensus_queue.cc:1035] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [LEADER]: Connected to new peer: Peer: permanent_uuid: "a0c0305a936046b6b6ced8b2d8b51e30" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:44.599246 11641 mvcc.cc:205] Tried to move back new op lower bound from 7298547239270674432 to 7298547239192154112. Current Snapshot: MvccSnapshot[applied={T|T < 7298547239270674432}]
I20260619 12:24:44.610242 11154 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:44.610442 11154 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:44.612986 11154 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 45ccc1140cef4b5f9410394c597cfe3b: Sending DeleteTablet for 3 replicas of tablet 6f82c842fff3419097c68ad733df402d
I20260619 12:24:44.613847 11544 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f82c842fff3419097c68ad733df402d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:44 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:60578
I20260619 12:24:44.613909 11278 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f82c842fff3419097c68ad733df402d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:44 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:50992
I20260619 12:24:44.613854 11411 tablet_service.cc:1514] Processing DeleteTablet for tablet 6f82c842fff3419097c68ad733df402d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:44 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:40254
I20260619 12:24:44.614408 11650 tablet_replica.cc:330] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7: stopping tablet replica
I20260619 12:24:44.614605 11652 tablet_replica.cc:330] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30: stopping tablet replica
I20260619 12:24:44.614837 11650 raft_consensus.cc:2229] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:44.614948 11652 raft_consensus.cc:2229] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:44.615284 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 11216
I20260619 12:24:44.615283 11650 raft_consensus.cc:2258] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:44.615283 11652 raft_consensus.cc:2258] T 6f82c842fff3419097c68ad733df402d P a0c0305a936046b6b6ced8b2d8b51e30 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:44.615718 11651 tablet_replica.cc:330] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab: stopping tablet replica
I20260619 12:24:44.616073 11651 raft_consensus.cc:2229] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:24:44.616523 11651 raft_consensus.cc:2258] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:44.617667 11650 ts_tablet_manager.cc:1905] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:44.620537 11650 ts_tablet_manager.cc:1918] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:44.620641 11650 log.cc:1196] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/6f82c842fff3419097c68ad733df402d
I20260619 12:24:44.620975 11650 ts_tablet_manager.cc:1939] T 6f82c842fff3419097c68ad733df402d P 0da017ab35144c628450472df2a990f7: Deleting consensus metadata
I20260619 12:24:44.622435 11139 catalog_manager.cc:4841] TS 0da017ab35144c628450472df2a990f7 (127.6.231.194:38115): tablet 6f82c842fff3419097c68ad733df402d (table test-table [id=7e5b47e761ba4cd5bb2676403ee80374]) successfully deleted
W20260619 12:24:44.622789 11141 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:45257 (error 108)
I20260619 12:24:44.623725 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 11349
I20260619 12:24:44.623826 11651 ts_tablet_manager.cc:1905] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:44.624564 11621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.623594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:45257 (local address 127.0.0.1:51000)
0619 12:24:44.624357 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:44.624448 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:45257: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":640}
W20260619 12:24:44.625037 11141 catalog_manager.cc:4584] TS a0c0305a936046b6b6ced8b2d8b51e30 (127.6.231.193:45257): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6f82c842fff3419097c68ad733df402d: Network error: Client connection negotiation failed: client connection to 127.6.231.193:45257: connect: Connection refused (error 111)
I20260619 12:24:44.626474 11651 ts_tablet_manager.cc:1918] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:44.626576 11651 log.cc:1196] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/6f82c842fff3419097c68ad733df402d
I20260619 12:24:44.626884 11651 ts_tablet_manager.cc:1939] T 6f82c842fff3419097c68ad733df402d P 51840727ccdd48b19491c84fd65feeab: Deleting consensus metadata
I20260619 12:24:44.628170 11140 catalog_manager.cc:4841] TS 51840727ccdd48b19491c84fd65feeab (127.6.231.195:37935): tablet 6f82c842fff3419097c68ad733df402d (table test-table [id=7e5b47e761ba4cd5bb2676403ee80374]) successfully deleted
I20260619 12:24:44.630906 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 11482
I20260619 12:24:44.635598 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 11123
2026-06-19T12:24:44Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2355 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-06-19T12:24:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:44Z Disabled control of system clock
I20260619 12:24:44.668323 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:40181
--webserver_interface=127.6.231.254
--webserver_port=0
--builtin_ntp_servers=127.6.231.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:40181
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 12:24:44.783264 11657 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:44.783535 11657 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:44.783572 11657 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:44.787685 11657 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:44.787760 11657 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:44.787776 11657 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:44.787811 11657 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:44.787832 11657 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:44.793186 11657 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:40181
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:40181
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11657
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:44.794402 11657 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:44.795749 11657 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:44.804410 11665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:44.804652 11657 server_base.cc:1034] running on GCE node
W20260619 12:24:44.804749 11662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:44.804643 11663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:44.805565 11657 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:44.806844 11657 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:44.808027 11657 hybrid_clock.cc:648] HybridClock initialized: now 1781871884808007 us; error 36 us; skew 500 ppm
I20260619 12:24:44.814246 11657 webserver.cc:460] Webserver started at http://127.6.231.254:37987/ using document root <none> and password file <none>
I20260619 12:24:44.814903 11657 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:44.814970 11657 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:44.815204 11657 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:44.817402 11657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "900742860c9647abbb006d4df96fb7bb"
format_stamp: "Formatted at 2026-06-19 12:24:44 on dist-test-slave-kr9h"
server_key: "3a43f9dd5dcb7aaa0e3e64d9eb3aad17"
server_key_iv: "96dab9ece49b65ce91c26d43e0e47d52"
server_key_version: "encryptionkey@0"
I20260619 12:24:44.818073 11657 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "900742860c9647abbb006d4df96fb7bb"
format_stamp: "Formatted at 2026-06-19 12:24:44 on dist-test-slave-kr9h"
server_key: "3a43f9dd5dcb7aaa0e3e64d9eb3aad17"
server_key_iv: "96dab9ece49b65ce91c26d43e0e47d52"
server_key_version: "encryptionkey@0"
I20260619 12:24:44.821988 11657 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:24:44.824491 11671 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:44.825963 11657 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:44.826081 11657 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "900742860c9647abbb006d4df96fb7bb"
format_stamp: "Formatted at 2026-06-19 12:24:44 on dist-test-slave-kr9h"
server_key: "3a43f9dd5dcb7aaa0e3e64d9eb3aad17"
server_key_iv: "96dab9ece49b65ce91c26d43e0e47d52"
server_key_version: "encryptionkey@0"
I20260619 12:24:44.826182 11657 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:44.831146 11657 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:44.831796 11657 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:44.831954 11657 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:44.839807 11657 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:40181
I20260619 12:24:44.839838 11723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:40181 every 8 connection(s)
I20260619 12:24:44.840821 11657 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:44.844249 11724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:44.844432 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 11657
I20260619 12:24:44.844591 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:44.844870 7071 external_mini_cluster.cc:1428] Setting key 1069d3f777e1508024144ef3c110873d
I20260619 12:24:44.850761 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:44.846341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33686 (local address 127.6.231.254:40181)
0619 12:24:44.846985 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:44.847001 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:44.847037 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:24:44.847727 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:44.847745 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:44.848107 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:44.848403 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:44.849320 (+ 917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:44.849325 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:44.849337 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:44.849348 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:44.849490 (+ 142us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:44.849513 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:44.849630 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:44.849754 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 12:24:44.850010 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":132,"threads_started":1}
I20260619 12:24:44.851665 11724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb: Bootstrap starting.
I20260619 12:24:44.854040 11724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:44.854732 11724 log.cc:824] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:44.856930 11724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb: No bootstrap required, opened a new log
I20260619 12:24:44.860441 11724 raft_consensus.cc:348] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900742860c9647abbb006d4df96fb7bb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 40181 } }
I20260619 12:24:44.860661 11724 raft_consensus.cc:374] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:44.860695 11724 raft_consensus.cc:729] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 900742860c9647abbb006d4df96fb7bb, State: Initialized, Role: FOLLOWER
I20260619 12:24:44.861225 11724 consensus_queue.cc:260] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900742860c9647abbb006d4df96fb7bb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 40181 } }
I20260619 12:24:44.861405 11724 raft_consensus.cc:388] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:44.861490 11724 raft_consensus.cc:482] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:44.861611 11724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:44.862753 11724 raft_consensus.cc:504] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900742860c9647abbb006d4df96fb7bb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 40181 } }
I20260619 12:24:44.863133 11724 leader_election.cc:302] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 900742860c9647abbb006d4df96fb7bb; no voters:
I20260619 12:24:44.863583 11724 leader_election.cc:288] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:44.863806 11729 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:44.864053 11729 raft_consensus.cc:686] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [term 1 LEADER]: Becoming Leader. State: Replica: 900742860c9647abbb006d4df96fb7bb, State: Running, Role: LEADER
I20260619 12:24:44.864391 11729 consensus_queue.cc:237] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900742860c9647abbb006d4df96fb7bb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 40181 } }
I20260619 12:24:44.865042 11724 sys_catalog.cc:564] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:44.866375 11730 sys_catalog.cc:455] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "900742860c9647abbb006d4df96fb7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900742860c9647abbb006d4df96fb7bb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 40181 } } }
I20260619 12:24:44.866396 11731 sys_catalog.cc:455] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 900742860c9647abbb006d4df96fb7bb. Latest consensus state: current_term: 1 leader_uuid: "900742860c9647abbb006d4df96fb7bb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900742860c9647abbb006d4df96fb7bb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 40181 } } }
I20260619 12:24:44.866524 11730 sys_catalog.cc:458] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:44.866524 11731 sys_catalog.cc:458] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:44.866997 11738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:44.871004 11738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:44.876991 11738 catalog_manager.cc:1347] Generated new cluster ID: e0e5e71c5efe4e9c879aa0bde3ddb273
I20260619 12:24:44.877085 11738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:44.889923 11738 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:44.890240 11738 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:44.894969 11738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb: Generated new TSK 0
I20260619 12:24:44.895663 11738 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:24:44.903854 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:40181
--builtin_ntp_servers=127.6.231.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 12:24:45.016381 11748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:45.016654 11748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:45.016695 11748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:45.020850 11748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:45.020905 11748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:45.020987 11748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:24:45.025748 11748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11748
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:45.026975 11748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:45.028335 11748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:45.038210 11753 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:45.038218 11754 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:45.038223 11756 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:45.038568 11748 server_base.cc:1034] running on GCE node
I20260619 12:24:45.039044 11748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:45.039723 11748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:45.040990 11748 hybrid_clock.cc:648] HybridClock initialized: now 1781871885040959 us; error 42 us; skew 500 ppm
I20260619 12:24:45.043076 11748 webserver.cc:460] Webserver started at http://127.6.231.193:39327/ using document root <none> and password file <none>
I20260619 12:24:45.043713 11748 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:45.043785 11748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:45.044102 11748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:45.046319 11748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "2e5b3f244de540aca42e64d20e49a28f"
format_stamp: "Formatted at 2026-06-19 12:24:45 on dist-test-slave-kr9h"
server_key: "36b6cc54d3e4efc3106265ad49d56663"
server_key_iv: "03c881b31c2a6b53a0c1164bbf6cea3b"
server_key_version: "encryptionkey@0"
I20260619 12:24:45.046914 11748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "2e5b3f244de540aca42e64d20e49a28f"
format_stamp: "Formatted at 2026-06-19 12:24:45 on dist-test-slave-kr9h"
server_key: "36b6cc54d3e4efc3106265ad49d56663"
server_key_iv: "03c881b31c2a6b53a0c1164bbf6cea3b"
server_key_version: "encryptionkey@0"
I20260619 12:24:45.050806 11748 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:24:45.053529 11762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:45.054849 11748 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 12:24:45.055001 11748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "2e5b3f244de540aca42e64d20e49a28f"
format_stamp: "Formatted at 2026-06-19 12:24:45 on dist-test-slave-kr9h"
server_key: "36b6cc54d3e4efc3106265ad49d56663"
server_key_iv: "03c881b31c2a6b53a0c1164bbf6cea3b"
server_key_version: "encryptionkey@0"
I20260619 12:24:45.055122 11748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:45.060228 11748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:45.061014 11748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:45.061219 11748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:45.061885 11748 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:45.063023 11748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:45.063094 11748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:45.063156 11748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:45.063185 11748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:45.072283 11748 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:35183
I20260619 12:24:45.072322 11875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:35183 every 8 connection(s)
I20260619 12:24:45.073252 11748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:24:45.079612 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.075198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:59289 (local address 127.6.231.254:40181)
0619 12:24:45.075336 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:45.075340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:45.076456 (+ 1116us) server_negotiation.cc:407] Connection header received
0619 12:24:45.077865 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:45.077868 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:45.077912 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:45.077992 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:45.079115 (+ 1123us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:45.079118 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:45.079121 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:45.079142 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:45.079161 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:45.079170 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:45.079277 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:45.079424 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 12:24:45.079472 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:24:45.080246 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 11748
I20260619 12:24:45.080375 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:24:45.080576 11878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.075598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:40181 (local address 127.6.231.193:59289)
0619 12:24:45.076272 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:45.076330 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:45.077644 (+ 1314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:45.078152 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:45.078159 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:45.078251 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:45.078839 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:24:45.078840 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:45.078881 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:24:45.078952 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:45.079257 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:45.079266 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:45.079272 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:24:45.079469 (+ 197us) client_negotiation.cc:240] Negotiation successful
0619 12:24:45.079709 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":103,"threads_started":1}
I20260619 12:24:45.080690 7071 external_mini_cluster.cc:1428] Setting key 1c9ce67ef9cec5e93a484f8763ff4c49
I20260619 12:24:45.081908 11876 heartbeater.cc:347] Connected to a master server at 127.6.231.254:40181
I20260619 12:24:45.082160 11876 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:45.082587 11876 heartbeater.cc:511] Master 127.6.231.254:40181 requested a full tablet report, sending...
I20260619 12:24:45.082988 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:40181
--builtin_ntp_servers=127.6.231.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260619 12:24:45.084127 11687 ts_manager.cc:194] Registered new tserver with Master: 2e5b3f244de540aca42e64d20e49a28f (127.6.231.193:35183)
W20260619 12:24:45.194422 11879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:45.194659 11879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:45.194697 11879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:45.198772 11879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:45.198823 11879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:45.198880 11879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:24:45.203545 11879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:45.204557 11879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:45.205904 11879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:45.215353 11887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:45.215342 11885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:45.215605 11879 server_base.cc:1034] running on GCE node
W20260619 12:24:45.215337 11884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:45.216043 11879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:45.216732 11879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:45.217918 11879 hybrid_clock.cc:648] HybridClock initialized: now 1781871885217908 us; error 33 us; skew 500 ppm
I20260619 12:24:45.219780 11879 webserver.cc:460] Webserver started at http://127.6.231.194:45845/ using document root <none> and password file <none>
I20260619 12:24:45.220427 11879 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:45.220520 11879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:45.220754 11879 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:45.222842 11879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "bf8f5506888d450482dbc694d2517d0a"
format_stamp: "Formatted at 2026-06-19 12:24:45 on dist-test-slave-kr9h"
server_key: "ba94278b61bddcf91c8bbc56043f10d5"
server_key_iv: "6b61d374f677d18e06112e8b958ae6bf"
server_key_version: "encryptionkey@0"
I20260619 12:24:45.223404 11879 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "bf8f5506888d450482dbc694d2517d0a"
format_stamp: "Formatted at 2026-06-19 12:24:45 on dist-test-slave-kr9h"
server_key: "ba94278b61bddcf91c8bbc56043f10d5"
server_key_iv: "6b61d374f677d18e06112e8b958ae6bf"
server_key_version: "encryptionkey@0"
I20260619 12:24:45.227088 11879 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 12:24:45.229616 11893 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:45.230944 11879 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:24:45.231065 11879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "bf8f5506888d450482dbc694d2517d0a"
format_stamp: "Formatted at 2026-06-19 12:24:45 on dist-test-slave-kr9h"
server_key: "ba94278b61bddcf91c8bbc56043f10d5"
server_key_iv: "6b61d374f677d18e06112e8b958ae6bf"
server_key_version: "encryptionkey@0"
I20260619 12:24:45.231189 11879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:45.238096 11879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:45.238837 11879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:45.239012 11879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:45.239691 11879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:45.240759 11879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:45.240803 11879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:45.240849 11879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:45.240867 11879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:45.249236 11879 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:36979
I20260619 12:24:45.249282 12006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:36979 every 8 connection(s)
I20260619 12:24:45.250202 11879 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:24:45.256551 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.252014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:40281 (local address 127.6.231.254:40181)
0619 12:24:45.252149 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:45.252153 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:45.253360 (+ 1207us) server_negotiation.cc:407] Connection header received
0619 12:24:45.254771 (+ 1411us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:45.254773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:45.254870 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:45.254954 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:45.256105 (+ 1151us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:45.256108 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:45.256110 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:45.256122 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:45.256141 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:45.256150 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:45.256241 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:45.256403 (+ 162us) server_negotiation.cc:299] Negotiation successful
0619 12:24:45.256440 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:24:45.257540 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.252407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:40181 (local address 127.6.231.194:40281)
0619 12:24:45.253196 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:45.253239 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:45.254548 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:45.255126 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:45.255133 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:45.255224 (+ 91us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:45.255824 (+ 600us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:24:45.255826 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:45.255868 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:24:45.255936 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:45.256262 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:45.256270 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:45.256275 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:24:45.256447 (+ 172us) client_negotiation.cc:240] Negotiation successful
0619 12:24:45.256689 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":105,"threads_started":1}
I20260619 12:24:45.258781 12007 heartbeater.cc:347] Connected to a master server at 127.6.231.254:40181
I20260619 12:24:45.259063 12007 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:45.259487 12007 heartbeater.cc:511] Master 127.6.231.254:40181 requested a full tablet report, sending...
I20260619 12:24:45.259733 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 11879
I20260619 12:24:45.259850 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:24:45.260210 7071 external_mini_cluster.cc:1428] Setting key 90be0da14b97f6d336a1967c2e153aff
I20260619 12:24:45.260675 11687 ts_manager.cc:194] Registered new tserver with Master: bf8f5506888d450482dbc694d2517d0a (127.6.231.194:36979)
I20260619 12:24:45.262171 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:40181
--builtin_ntp_servers=127.6.231.212:45457
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260619 12:24:45.372992 12010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:45.373291 12010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:45.373381 12010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:45.377388 12010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:45.377494 12010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:45.377592 12010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:24:45.382385 12010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:45457
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:40181
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:45.383540 12010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:45.384946 12010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:45.394642 12015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:45.394726 12018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:45.394946 12010 server_base.cc:1034] running on GCE node
W20260619 12:24:45.395023 12016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:45.395530 12010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:45.396245 12010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:45.397424 12010 hybrid_clock.cc:648] HybridClock initialized: now 1781871885397412 us; error 36 us; skew 500 ppm
I20260619 12:24:45.399370 12010 webserver.cc:460] Webserver started at http://127.6.231.195:32963/ using document root <none> and password file <none>
I20260619 12:24:45.399899 12010 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:45.399967 12010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:45.400131 12010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:45.402347 12010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "d78fcd7d728049ebb4897cfb01edd6ad"
format_stamp: "Formatted at 2026-06-19 12:24:45 on dist-test-slave-kr9h"
server_key: "f26bd09a88a1454a66df489731f18b48"
server_key_iv: "9f96f52c207ae1eef2138184b7b15fdf"
server_key_version: "encryptionkey@0"
I20260619 12:24:45.402860 12010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "d78fcd7d728049ebb4897cfb01edd6ad"
format_stamp: "Formatted at 2026-06-19 12:24:45 on dist-test-slave-kr9h"
server_key: "f26bd09a88a1454a66df489731f18b48"
server_key_iv: "9f96f52c207ae1eef2138184b7b15fdf"
server_key_version: "encryptionkey@0"
I20260619 12:24:45.406570 12010 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.003s
I20260619 12:24:45.409022 12024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:45.410302 12010 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 12:24:45.410426 12010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "d78fcd7d728049ebb4897cfb01edd6ad"
format_stamp: "Formatted at 2026-06-19 12:24:45 on dist-test-slave-kr9h"
server_key: "f26bd09a88a1454a66df489731f18b48"
server_key_iv: "9f96f52c207ae1eef2138184b7b15fdf"
server_key_version: "encryptionkey@0"
I20260619 12:24:45.410538 12010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:45.415215 12010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:45.415791 12010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:45.415972 12010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:45.416501 12010 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:45.417363 12010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:45.417408 12010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:45.417503 12010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:45.417522 12010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:45.426028 12010 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:39059
I20260619 12:24:45.426055 12137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:39059 every 8 connection(s)
I20260619 12:24:45.426918 12010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:24:45.428081 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 12010
I20260619 12:24:45.428211 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:24:45.428696 7071 external_mini_cluster.cc:1428] Setting key d841fab0a28b6f604cf562bd1bdba162
I20260619 12:24:45.433761 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.429163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:57277 (local address 127.6.231.254:40181)
0619 12:24:45.429291 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:45.429295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:45.430631 (+ 1336us) server_negotiation.cc:407] Connection header received
0619 12:24:45.432019 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:45.432022 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:45.432060 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:45.432136 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:45.433309 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:45.433313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:45.433315 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:45.433334 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:45.433355 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:45.433363 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:45.433505 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:45.433618 (+ 113us) server_negotiation.cc:299] Negotiation successful
0619 12:24:45.433656 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260619 12:24:45.434629 12140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.429675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:40181 (local address 127.6.231.195:57277)
0619 12:24:45.430466 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:45.430522 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:45.431795 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:45.432276 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:45.432283 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:45.432372 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:45.433030 (+ 658us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:24:45.433032 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:45.433074 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:24:45.433146 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:45.433476 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:45.433484 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:45.433490 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:24:45.433637 (+ 147us) client_negotiation.cc:240] Negotiation successful
0619 12:24:45.433830 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":108,"threads_started":1}
I20260619 12:24:45.436029 12138 heartbeater.cc:347] Connected to a master server at 127.6.231.254:40181
I20260619 12:24:45.436321 12138 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:45.436755 12138 heartbeater.cc:511] Master 127.6.231.254:40181 requested a full tablet report, sending...
I20260619 12:24:45.437949 11688 ts_manager.cc:194] Registered new tserver with Master: d78fcd7d728049ebb4897cfb01edd6ad (127.6.231.195:39059)
I20260619 12:24:45.442556 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:24:45.445989 11727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.444409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33696 (local address 127.6.231.254:40181)
0619 12:24:45.444536 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:45.444540 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:45.444733 (+ 193us) server_negotiation.cc:407] Connection header received
0619 12:24:45.444830 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:45.444833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:45.444865 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:45.444934 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:45.445682 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:45.445685 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:45.445686 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:45.445697 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:45.445716 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:45.445724 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:45.445807 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:45.445856 (+ 49us) server_negotiation.cc:299] Negotiation successful
0619 12:24:45.445889 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 12:24:45.449039 11688 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:24:45.452282 11688 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:45.464583 12151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.462045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:39059 (local address 127.0.0.1:46598)
0619 12:24:45.462280 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:45.462294 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:45.462375 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:45.463295 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:45.463300 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:45.463323 (+ 23us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:45.463832 (+ 509us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:24:45.463834 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:45.463844 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:24:45.463879 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:45.464224 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:45.464241 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:45.464247 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:24:45.464372 (+ 125us) client_negotiation.cc:240] Negotiation successful
0619 12:24:45.464445 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":105,"threads_started":1}
I20260619 12:24:45.464757 12153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.462326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46598 (local address 127.6.231.195:39059)
0619 12:24:45.462871 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:45.462887 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:45.462914 (+ 27us) server_negotiation.cc:407] Connection header received
0619 12:24:45.462966 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:45.462971 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:45.463144 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:45.463287 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:45.464024 (+ 737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:45.464030 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:45.464039 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:45.464053 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:45.464092 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:45.464128 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:45.464241 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:45.464412 (+ 171us) server_negotiation.cc:299] Negotiation successful
0619 12:24:45.464494 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":425,"thread_start_us":155,"threads_started":1}
I20260619 12:24:45.465107 12150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.461398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38568 (local address 127.6.231.193:35183)
0619 12:24:45.461893 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:45.461910 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:45.461947 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:24:45.462171 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:45.462179 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:45.462409 (+ 230us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:45.462556 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:45.463930 (+ 1374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:45.463939 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:45.463956 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:45.463970 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:45.464012 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:45.464039 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:45.464449 (+ 410us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:45.464837 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 12:24:45.464914 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":385,"thread_start_us":126,"threads_started":1}
I20260619 12:24:45.465337 12149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.461263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:35183 (local address 127.0.0.1:38568)
0619 12:24:45.461679 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:45.461736 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:45.462009 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:45.462542 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:45.462549 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:45.462631 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:45.463574 (+ 943us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:24:45.463577 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:45.463601 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:24:45.463678 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:45.464491 (+ 813us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:45.464494 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:45.464497 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:24:45.464588 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 12:24:45.465042 (+ 454us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":99,"threads_started":1}
I20260619 12:24:45.465970 12152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.462408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:36979 (local address 127.0.0.1:40170)
0619 12:24:45.463123 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:45.463136 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:45.463212 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:45.463905 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:45.463907 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:45.463921 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:45.464887 (+ 966us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:24:45.464890 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:45.464900 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:24:45.464925 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:45.465722 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:45.465728 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:45.465732 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:24:45.465823 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 12:24:45.465857 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":656,"thread_start_us":86,"threads_started":1}
I20260619 12:24:45.466097 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.462783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40170 (local address 127.6.231.194:36979)
0619 12:24:45.463445 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:45.463455 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:45.463477 (+ 22us) server_negotiation.cc:407] Connection header received
0619 12:24:45.463528 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:45.463537 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:45.463698 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:45.463821 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:45.465290 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:45.465296 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:45.465315 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:45.465333 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:45.465367 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:45.465386 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:45.465565 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:45.465832 (+ 267us) server_negotiation.cc:299] Negotiation successful
0619 12:24:45.465895 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":106,"threads_started":1}
I20260619 12:24:45.468010 11810 tablet_service.cc:1467] Processing CreateTablet for tablet 4eb85da9ca02440c8283c3571be060cc (DEFAULT_TABLE table=test-table [id=7336c4727e8d4344963cb640452e16b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:45.467962 12072 tablet_service.cc:1467] Processing CreateTablet for tablet 4eb85da9ca02440c8283c3571be060cc (DEFAULT_TABLE table=test-table [id=7336c4727e8d4344963cb640452e16b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:45.468909 11810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eb85da9ca02440c8283c3571be060cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:45.468909 12072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eb85da9ca02440c8283c3571be060cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:45.472005 11941 tablet_service.cc:1467] Processing CreateTablet for tablet 4eb85da9ca02440c8283c3571be060cc (DEFAULT_TABLE table=test-table [id=7336c4727e8d4344963cb640452e16b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:45.473116 11941 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4eb85da9ca02440c8283c3571be060cc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:45.475486 12155 tablet_bootstrap.cc:492] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad: Bootstrap starting.
I20260619 12:24:45.476523 12156 tablet_bootstrap.cc:492] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f: Bootstrap starting.
I20260619 12:24:45.477926 12155 tablet_bootstrap.cc:654] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:45.478724 12156 tablet_bootstrap.cc:654] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:45.478945 12155 log.cc:824] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:45.479517 12156 log.cc:824] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:45.480165 12159 tablet_bootstrap.cc:492] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a: Bootstrap starting.
I20260619 12:24:45.481122 12155 tablet_bootstrap.cc:492] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad: No bootstrap required, opened a new log
I20260619 12:24:45.481329 12155 ts_tablet_manager.cc:1397] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 12:24:45.481336 12156 tablet_bootstrap.cc:492] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f: No bootstrap required, opened a new log
I20260619 12:24:45.481571 12156 ts_tablet_manager.cc:1397] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260619 12:24:45.483547 12159 tablet_bootstrap.cc:654] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:45.484480 12159 log.cc:824] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:45.485689 12155 raft_consensus.cc:348] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e5b3f244de540aca42e64d20e49a28f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35183 } } peers { permanent_uuid: "d78fcd7d728049ebb4897cfb01edd6ad" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39059 } } peers { permanent_uuid: "bf8f5506888d450482dbc694d2517d0a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 36979 } }
I20260619 12:24:45.485936 12155 raft_consensus.cc:374] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:45.485993 12155 raft_consensus.cc:729] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d78fcd7d728049ebb4897cfb01edd6ad, State: Initialized, Role: FOLLOWER
I20260619 12:24:45.485930 12156 raft_consensus.cc:348] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e5b3f244de540aca42e64d20e49a28f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35183 } } peers { permanent_uuid: "d78fcd7d728049ebb4897cfb01edd6ad" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39059 } } peers { permanent_uuid: "bf8f5506888d450482dbc694d2517d0a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 36979 } }
I20260619 12:24:45.486187 12156 raft_consensus.cc:374] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:45.486249 12156 raft_consensus.cc:729] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e5b3f244de540aca42e64d20e49a28f, State: Initialized, Role: FOLLOWER
I20260619 12:24:45.486464 12159 tablet_bootstrap.cc:492] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a: No bootstrap required, opened a new log
I20260619 12:24:45.486634 12159 ts_tablet_manager.cc:1397] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260619 12:24:45.486567 12155 consensus_queue.cc:260] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e5b3f244de540aca42e64d20e49a28f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35183 } } peers { permanent_uuid: "d78fcd7d728049ebb4897cfb01edd6ad" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39059 } } peers { permanent_uuid: "bf8f5506888d450482dbc694d2517d0a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 36979 } }
I20260619 12:24:45.486676 12156 consensus_queue.cc:260] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e5b3f244de540aca42e64d20e49a28f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35183 } } peers { permanent_uuid: "d78fcd7d728049ebb4897cfb01edd6ad" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39059 } } peers { permanent_uuid: "bf8f5506888d450482dbc694d2517d0a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 36979 } }
I20260619 12:24:45.487229 12138 heartbeater.cc:503] Master 127.6.231.254:40181 was elected leader, sending a full tablet report...
I20260619 12:24:45.487584 12155 ts_tablet_manager.cc:1428] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 12:24:45.487584 12156 ts_tablet_manager.cc:1428] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 12:24:45.488121 11876 heartbeater.cc:503] Master 127.6.231.254:40181 was elected leader, sending a full tablet report...
I20260619 12:24:45.490893 12159 raft_consensus.cc:348] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e5b3f244de540aca42e64d20e49a28f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35183 } } peers { permanent_uuid: "d78fcd7d728049ebb4897cfb01edd6ad" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39059 } } peers { permanent_uuid: "bf8f5506888d450482dbc694d2517d0a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 36979 } }
I20260619 12:24:45.491147 12159 raft_consensus.cc:374] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:45.491204 12159 raft_consensus.cc:729] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf8f5506888d450482dbc694d2517d0a, State: Initialized, Role: FOLLOWER
I20260619 12:24:45.491600 12159 consensus_queue.cc:260] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e5b3f244de540aca42e64d20e49a28f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35183 } } peers { permanent_uuid: "d78fcd7d728049ebb4897cfb01edd6ad" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39059 } } peers { permanent_uuid: "bf8f5506888d450482dbc694d2517d0a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 36979 } }
I20260619 12:24:45.492419 12007 heartbeater.cc:503] Master 127.6.231.254:40181 was elected leader, sending a full tablet report...
I20260619 12:24:45.492667 12159 ts_tablet_manager.cc:1428] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260619 12:24:45.501135 12008 tablet.cc:2369] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:24:45.501672 12163 raft_consensus.cc:482] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:45.501829 12163 raft_consensus.cc:504] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e5b3f244de540aca42e64d20e49a28f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35183 } } peers { permanent_uuid: "d78fcd7d728049ebb4897cfb01edd6ad" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39059 } } peers { permanent_uuid: "bf8f5506888d450482dbc694d2517d0a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 36979 } }
I20260619 12:24:45.502761 12163 leader_election.cc:288] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e5b3f244de540aca42e64d20e49a28f (127.6.231.193:35183), d78fcd7d728049ebb4897cfb01edd6ad (127.6.231.195:39059)
I20260619 12:24:45.504617 12150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.503161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:55919 (local address 127.6.231.193:35183)
0619 12:24:45.503283 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:45.503287 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:45.503299 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:24:45.503341 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:45.503343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:45.503372 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:45.503481 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:45.504253 (+ 772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:45.504259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:45.504266 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:45.504278 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:45.504295 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:45.504303 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:45.504397 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:45.504462 (+ 65us) server_negotiation.cc:299] Negotiation successful
0619 12:24:45.504500 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:24:45.504617 12009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.503033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:35183 (local address 127.6.231.194:55919)
0619 12:24:45.503158 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:45.503175 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:45.503245 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:45.503505 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:45.503508 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:45.503525 (+ 17us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:45.504109 (+ 584us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:24:45.504111 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:45.504120 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:24:45.504141 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:45.504384 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:45.504387 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:45.504390 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:24:45.504467 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 12:24:45.504500 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 12:24:45.505050 12164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.503292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:39059 (local address 127.6.231.194:37029)
0619 12:24:45.503620 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:45.503640 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:45.503735 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:45.503985 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:45.503989 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:45.504005 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:45.504455 (+ 450us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:24:45.504457 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:45.504467 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:24:45.504499 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:45.504813 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:45.504815 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:45.504818 (+ 3us) client_negotiation.cc:769] Sending connection context
0619 12:24:45.504890 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 12:24:45.504920 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":107,"threads_started":1}
I20260619 12:24:45.505084 12153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.503533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:37029 (local address 127.6.231.195:39059)
0619 12:24:45.503678 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:45.503685 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:45.503710 (+ 25us) server_negotiation.cc:407] Connection header received
0619 12:24:45.503829 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:45.503831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:45.503869 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:45.503969 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:45.504614 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:45.504617 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:45.504620 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:45.504631 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:45.504649 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:45.504678 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:45.504771 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:45.504922 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 12:24:45.504961 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:24:45.505259 11830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eb85da9ca02440c8283c3571be060cc" candidate_uuid: "bf8f5506888d450482dbc694d2517d0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e5b3f244de540aca42e64d20e49a28f" is_pre_election: true
I20260619 12:24:45.505580 11830 raft_consensus.cc:2454] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf8f5506888d450482dbc694d2517d0a in term 0.
I20260619 12:24:45.505920 12092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eb85da9ca02440c8283c3571be060cc" candidate_uuid: "bf8f5506888d450482dbc694d2517d0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d78fcd7d728049ebb4897cfb01edd6ad" is_pre_election: true
I20260619 12:24:45.506013 11894 leader_election.cc:302] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e5b3f244de540aca42e64d20e49a28f, bf8f5506888d450482dbc694d2517d0a; no voters:
I20260619 12:24:45.506199 12092 raft_consensus.cc:2454] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bf8f5506888d450482dbc694d2517d0a in term 0.
I20260619 12:24:45.506287 12163 raft_consensus.cc:2781] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:45.506368 12163 raft_consensus.cc:482] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:45.506420 12163 raft_consensus.cc:3037] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:45.507464 12163 raft_consensus.cc:504] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e5b3f244de540aca42e64d20e49a28f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35183 } } peers { permanent_uuid: "d78fcd7d728049ebb4897cfb01edd6ad" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39059 } } peers { permanent_uuid: "bf8f5506888d450482dbc694d2517d0a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 36979 } }
I20260619 12:24:45.507833 12163 leader_election.cc:288] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [CANDIDATE]: Term 1 election: Requested vote from peers 2e5b3f244de540aca42e64d20e49a28f (127.6.231.193:35183), d78fcd7d728049ebb4897cfb01edd6ad (127.6.231.195:39059)
I20260619 12:24:45.508268 11830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eb85da9ca02440c8283c3571be060cc" candidate_uuid: "bf8f5506888d450482dbc694d2517d0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e5b3f244de540aca42e64d20e49a28f"
I20260619 12:24:45.508392 11830 raft_consensus.cc:3037] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:45.509047 12092 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4eb85da9ca02440c8283c3571be060cc" candidate_uuid: "bf8f5506888d450482dbc694d2517d0a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d78fcd7d728049ebb4897cfb01edd6ad"
I20260619 12:24:45.509172 12092 raft_consensus.cc:3037] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:45.509703 11830 raft_consensus.cc:2454] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf8f5506888d450482dbc694d2517d0a in term 1.
I20260619 12:24:45.510002 11894 leader_election.cc:302] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e5b3f244de540aca42e64d20e49a28f, bf8f5506888d450482dbc694d2517d0a; no voters:
I20260619 12:24:45.510185 12163 raft_consensus.cc:2781] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:45.510399 12163 raft_consensus.cc:686] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [term 1 LEADER]: Becoming Leader. State: Replica: bf8f5506888d450482dbc694d2517d0a, State: Running, Role: LEADER
I20260619 12:24:45.510528 12092 raft_consensus.cc:2454] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bf8f5506888d450482dbc694d2517d0a in term 1.
I20260619 12:24:45.510753 12163 consensus_queue.cc:237] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e5b3f244de540aca42e64d20e49a28f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35183 } } peers { permanent_uuid: "d78fcd7d728049ebb4897cfb01edd6ad" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39059 } } peers { permanent_uuid: "bf8f5506888d450482dbc694d2517d0a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 36979 } }
I20260619 12:24:45.514076 11687 catalog_manager.cc:5495] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a reported cstate change: term changed from 0 to 1, leader changed from <none> to bf8f5506888d450482dbc694d2517d0a (127.6.231.194). New cstate: current_term: 1 leader_uuid: "bf8f5506888d450482dbc694d2517d0a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e5b3f244de540aca42e64d20e49a28f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d78fcd7d728049ebb4897cfb01edd6ad" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bf8f5506888d450482dbc694d2517d0a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 36979 } health_report { overall_health: HEALTHY } } }
I20260619 12:24:45.529803 12154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.528107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40176 (local address 127.6.231.194:36979)
0619 12:24:45.528239 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:45.528243 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:45.528256 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:24:45.528331 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:45.528333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:45.528399 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:45.528474 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:45.529244 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:45.529247 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:45.529249 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:45.529258 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:45.529275 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:45.529282 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:45.529369 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:45.529590 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 12:24:45.529632 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:24:45.535983 11830 raft_consensus.cc:1261] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f [term 1 FOLLOWER]: Refusing update from remote peer bf8f5506888d450482dbc694d2517d0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:45.535983 12092 raft_consensus.cc:1261] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad [term 1 FOLLOWER]: Refusing update from remote peer bf8f5506888d450482dbc694d2517d0a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:45.536739 12165 consensus_queue.cc:1035] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d78fcd7d728049ebb4897cfb01edd6ad" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:45.537011 12163 consensus_queue.cc:1035] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e5b3f244de540aca42e64d20e49a28f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:45.545497 12170 mvcc.cc:205] Tried to move back new op lower bound from 7298547243147898880 to 7298547243054886912. Current Snapshot: MvccSnapshot[applied={T|T < 7298547243147898880}]
I20260619 12:24:45.546060 12169 mvcc.cc:205] Tried to move back new op lower bound from 7298547243147898880 to 7298547243054886912. Current Snapshot: MvccSnapshot[applied={T|T < 7298547243147898880}]
I20260619 12:24:45.554380 11687 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:45.554571 11687 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:45.557106 11687 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 900742860c9647abbb006d4df96fb7bb: Sending DeleteTablet for 3 replicas of tablet 4eb85da9ca02440c8283c3571be060cc
I20260619 12:24:45.557938 11810 tablet_service.cc:1514] Processing DeleteTablet for tablet 4eb85da9ca02440c8283c3571be060cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:45 UTC) from {username='slave'} at 127.0.0.1:38568
I20260619 12:24:45.558104 12072 tablet_service.cc:1514] Processing DeleteTablet for tablet 4eb85da9ca02440c8283c3571be060cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:45 UTC) from {username='slave'} at 127.0.0.1:46598
I20260619 12:24:45.558228 11941 tablet_service.cc:1514] Processing DeleteTablet for tablet 4eb85da9ca02440c8283c3571be060cc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:45 UTC) from {username='slave'} at 127.0.0.1:40170
I20260619 12:24:45.558636 12180 tablet_replica.cc:330] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad: stopping tablet replica
I20260619 12:24:45.558703 12179 tablet_replica.cc:330] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f: stopping tablet replica
I20260619 12:24:45.559043 12179 raft_consensus.cc:2229] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:45.559051 12180 raft_consensus.cc:2229] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:45.559207 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 11748
I20260619 12:24:45.559362 12179 raft_consensus.cc:2258] T 4eb85da9ca02440c8283c3571be060cc P 2e5b3f244de540aca42e64d20e49a28f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:45.559602 12180 raft_consensus.cc:2258] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:45.559707 12181 tablet_replica.cc:330] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a: stopping tablet replica
I20260619 12:24:45.559966 12181 raft_consensus.cc:2229] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:24:45.560343 12181 raft_consensus.cc:2258] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:45.561794 12181 ts_tablet_manager.cc:1905] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:45.561801 12180 ts_tablet_manager.cc:1905] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:45.564491 12181 ts_tablet_manager.cc:1918] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:45.564589 12181 log.cc:1196] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/4eb85da9ca02440c8283c3571be060cc
I20260619 12:24:45.564931 12181 ts_tablet_manager.cc:1939] T 4eb85da9ca02440c8283c3571be060cc P bf8f5506888d450482dbc694d2517d0a: Deleting consensus metadata
I20260619 12:24:45.565999 12180 ts_tablet_manager.cc:1918] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:45.566166 12180 log.cc:1196] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/4eb85da9ca02440c8283c3571be060cc
I20260619 12:24:45.566133 11672 catalog_manager.cc:4841] TS bf8f5506888d450482dbc694d2517d0a (127.6.231.194:36979): tablet 4eb85da9ca02440c8283c3571be060cc (table test-table [id=7336c4727e8d4344963cb640452e16b6]) successfully deleted
I20260619 12:24:45.566592 12180 ts_tablet_manager.cc:1939] T 4eb85da9ca02440c8283c3571be060cc P d78fcd7d728049ebb4897cfb01edd6ad: Deleting consensus metadata
I20260619 12:24:45.567924 11672 catalog_manager.cc:4841] TS d78fcd7d728049ebb4897cfb01edd6ad (127.6.231.195:39059): tablet 4eb85da9ca02440c8283c3571be060cc (table test-table [id=7336c4727e8d4344963cb640452e16b6]) successfully deleted
W20260619 12:24:45.569156 11672 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:35183 (error 108)
I20260619 12:24:45.570060 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 11879
I20260619 12:24:45.570302 12152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.569966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:35183 (local address 127.0.0.1:38584)
0619 12:24:45.570095 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:45.570183 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:35183: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260619 12:24:45.575927 11672 catalog_manager.cc:4584] TS 2e5b3f244de540aca42e64d20e49a28f (127.6.231.193:35183): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4eb85da9ca02440c8283c3571be060cc: Network error: Client connection negotiation failed: client connection to 127.6.231.193:35183: connect: Connection refused (error 111)
I20260619 12:24:45.576023 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 12010
I20260619 12:24:45.581579 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 11657
2026-06-19T12:24:45Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (943 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260619 12:24:45.602929 7071 mini_oidc.cc:152] Starting JWKS server
I20260619 12:24:45.604640 7071 webserver.cc:460] Webserver started at https://127.0.0.1:42625/ using document root <none> and password file <none>
I20260619 12:24:45.604853 7071 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 12:24:45.605147 7071 webserver.cc:460] Webserver started at http://127.0.0.1:43973/ using document root <none> and password file <none>
2026-06-19T12:24:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:45Z Disabled control of system clock
I20260619 12:24:45.614154 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:33889
--webserver_interface=127.6.231.254
--webserver_port=0
--builtin_ntp_servers=127.6.231.212:39161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:33889
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42625/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/testchainca.cert with env {}
W20260619 12:24:45.725346 12188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:45.725744 12188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:45.725831 12188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:45.729928 12188 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:45.730031 12188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:45.730057 12188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:45.730098 12188 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:45.730132 12188 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 12:24:45.730166 12188 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 12:24:45.730194 12188 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42625/jwks.json
I20260619 12:24:45.736075 12188 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:39161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:33889
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:33889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42625/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:45.737236 12188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:45.738391 12188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:45.747227 12196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:45.747324 12194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:45.747248 12193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:45.747458 12188 server_base.cc:1034] running on GCE node
I20260619 12:24:45.748194 12188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:45.749477 12188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:45.750679 12188 hybrid_clock.cc:648] HybridClock initialized: now 1781871885750641 us; error 48 us; skew 500 ppm
I20260619 12:24:45.752790 12188 webserver.cc:460] Webserver started at http://127.6.231.254:34525/ using document root <none> and password file <none>
I20260619 12:24:45.753381 12188 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:45.753521 12188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:45.753793 12188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:45.755911 12188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "ff40a9d7460b47a8adedc2fb2f75078c"
format_stamp: "Formatted at 2026-06-19 12:24:45 on dist-test-slave-kr9h"
server_key: "5ef3e164538a6fe0ddd43df68608a12f"
server_key_iv: "859cc6391fe2da1f931e6f00ff9165b2"
server_key_version: "encryptionkey@0"
I20260619 12:24:45.756501 12188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "ff40a9d7460b47a8adedc2fb2f75078c"
format_stamp: "Formatted at 2026-06-19 12:24:45 on dist-test-slave-kr9h"
server_key: "5ef3e164538a6fe0ddd43df68608a12f"
server_key_iv: "859cc6391fe2da1f931e6f00ff9165b2"
server_key_version: "encryptionkey@0"
I20260619 12:24:45.760278 12188 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.002s
I20260619 12:24:45.762862 12202 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:45.764254 12188 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:45.764369 12188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "ff40a9d7460b47a8adedc2fb2f75078c"
format_stamp: "Formatted at 2026-06-19 12:24:45 on dist-test-slave-kr9h"
server_key: "5ef3e164538a6fe0ddd43df68608a12f"
server_key_iv: "859cc6391fe2da1f931e6f00ff9165b2"
server_key_version: "encryptionkey@0"
I20260619 12:24:45.764568 12188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:45.787562 12188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:45.788340 12188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:45.788563 12188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:45.795946 12188 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:33889
I20260619 12:24:45.795957 12254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:33889 every 8 connection(s)
I20260619 12:24:45.796948 12188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:45.799952 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 12188
I20260619 12:24:45.800086 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:45.800388 7071 external_mini_cluster.cc:1428] Setting key 74d9cb4e79a045caf7fe17dcac228b05
I20260619 12:24:45.800801 12255 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:45.807588 12255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c: Bootstrap starting.
I20260619 12:24:45.809973 12255 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:45.810377 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.801988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35346 (local address 127.6.231.254:33889)
0619 12:24:45.802657 (+ 669us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:45.802676 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:45.802711 (+ 35us) server_negotiation.cc:407] Connection header received
0619 12:24:45.803604 (+ 893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:45.803625 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:45.804066 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:45.804454 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:45.804917 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:45.806042 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:45.806966 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:45.807466 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:45.807643 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:45.808569 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:45.808574 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:45.808587 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:45.808984 (+ 397us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:45.809048 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:45.809068 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:45.809180 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:45.809306 (+ 126us) server_negotiation.cc:299] Negotiation successful
0619 12:24:45.809589 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":143,"threads_started":1}
I20260619 12:24:45.810886 12255 log.cc:824] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:45.813107 12255 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c: No bootstrap required, opened a new log
I20260619 12:24:45.813499 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:24:45.817145 12255 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff40a9d7460b47a8adedc2fb2f75078c" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33889 } }
I20260619 12:24:45.817381 12255 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:45.817474 12255 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff40a9d7460b47a8adedc2fb2f75078c, State: Initialized, Role: FOLLOWER
I20260619 12:24:45.817946 12255 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff40a9d7460b47a8adedc2fb2f75078c" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33889 } }
I20260619 12:24:45.818089 12255 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:45.818150 12255 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:45.818240 12255 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:45.819183 12255 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff40a9d7460b47a8adedc2fb2f75078c" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33889 } }
I20260619 12:24:45.819532 12255 leader_election.cc:302] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff40a9d7460b47a8adedc2fb2f75078c; no voters:
I20260619 12:24:45.819809 12255 leader_election.cc:288] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:45.819974 12260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:45.820233 12260 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [term 1 LEADER]: Becoming Leader. State: Replica: ff40a9d7460b47a8adedc2fb2f75078c, State: Running, Role: LEADER
I20260619 12:24:45.820585 12260 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff40a9d7460b47a8adedc2fb2f75078c" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33889 } }
I20260619 12:24:45.821210 12255 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:45.822532 12261 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff40a9d7460b47a8adedc2fb2f75078c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff40a9d7460b47a8adedc2fb2f75078c" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33889 } } }
I20260619 12:24:45.822686 12261 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:45.822901 12262 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff40a9d7460b47a8adedc2fb2f75078c. Latest consensus state: current_term: 1 leader_uuid: "ff40a9d7460b47a8adedc2fb2f75078c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff40a9d7460b47a8adedc2fb2f75078c" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33889 } } }
I20260619 12:24:45.823072 12262 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:45.823150 12269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:45.827216 12269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:45.832892 12269 catalog_manager.cc:1347] Generated new cluster ID: efa1b216e46847c290a9c6f936b9a7f8
I20260619 12:24:45.832974 12269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:45.849704 12269 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:45.851079 12269 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:45.857605 12269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff40a9d7460b47a8adedc2fb2f75078c: Generated new TSK 0
I20260619 12:24:45.858484 12269 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 12:24:45.895490 12285 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260619 12:24:45.895622 12285 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 12:24:45.896162 12258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:45.878677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35350 (local address 127.6.231.254:33889)
0619 12:24:45.878826 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:45.878830 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:45.879105 (+ 275us) server_negotiation.cc:407] Connection header received
0619 12:24:45.879360 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:45.879364 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:45.879417 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:45.879522 (+ 105us) server_negotiation.cc:226] Negotiated authn=JWT
0619 12:24:45.879855 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:45.880396 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:45.881621 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:45.881862 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:45.881951 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:45.882313 (+ 362us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 12:24:45.895824 (+ 13511us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 12:24:45.896051 (+ 227us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35350: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":58}
W20260619 12:24:45.896251 12258 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35350: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260619 12:24:45.897689 12284 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:24:45.878601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33889 (local address 127.0.0.1:35350)
0619 12:24:45.879010 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:45.879027 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:45.879162 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:45.879557 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:45.879561 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:45.879568 (+ 7us) client_negotiation.cc:189] Negotiated authn=JWT
0619 12:24:45.879707 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:45.879714 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:45.880508 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:45.880512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:45.881507 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:45.881518 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:45.881986 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:45.881990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:45.882082 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:45.882135 (+ 53us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 12:24:45.896057 (+ 13922us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0619 12:24:45.896327 (+ 270us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.231.254:33889: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":120,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.6.231.254:33889: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42625/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260619 12:24:45.898806 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 12188
2026-06-19T12:24:45Z chronyd exiting
I20260619 12:24:45.936128 7071 test_util.cc:175] -----------------------------------------------
I20260619 12:24:45.936269 7071 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1781871857195028-7071-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (334 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260619 12:24:45.937484 7071 mini_oidc.cc:152] Starting JWKS server
I20260619 12:24:45.938421 7071 webserver.cc:460] Webserver started at https://127.0.0.1:35235/ using document root <none> and password file <none>
I20260619 12:24:45.938568 7071 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 12:24:45.938860 7071 webserver.cc:460] Webserver started at http://127.0.0.1:37255/ using document root <none> and password file <none>
2026-06-19T12:24:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:45Z Disabled control of system clock
I20260619 12:24:45.948227 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:34943
--webserver_interface=127.6.231.254
--webserver_port=0
--builtin_ntp_servers=127.6.231.212:38687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:34943
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35235/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/testchainca.cert with env {}
W20260619 12:24:46.057852 12291 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:46.058116 12291 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:46.058157 12291 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:46.062233 12291 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:46.062297 12291 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:46.062314 12291 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:46.062330 12291 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:46.062345 12291 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 12:24:46.062393 12291 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 12:24:46.062414 12291 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35235/jwks.json
I20260619 12:24:46.069136 12291 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:34943
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:34943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35235/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12291
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:46.070223 12291 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:46.071211 12291 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:46.079756 12299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:46.079841 12297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:46.079761 12296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:46.080024 12291 server_base.cc:1034] running on GCE node
I20260619 12:24:46.080628 12291 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:46.081848 12291 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:46.083042 12291 hybrid_clock.cc:648] HybridClock initialized: now 1781871886083026 us; error 33 us; skew 500 ppm
I20260619 12:24:46.085062 12291 webserver.cc:460] Webserver started at http://127.6.231.254:45079/ using document root <none> and password file <none>
I20260619 12:24:46.085752 12291 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:46.085841 12291 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:46.086041 12291 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:46.088114 12291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "6cb122db71724c79b442a06354f89ce6"
format_stamp: "Formatted at 2026-06-19 12:24:46 on dist-test-slave-kr9h"
server_key: "8a3f77ba15eaee15b5459d5f53b5ed83"
server_key_iv: "6e6beaafbc746ace188ef811a8939846"
server_key_version: "encryptionkey@0"
I20260619 12:24:46.088776 12291 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "6cb122db71724c79b442a06354f89ce6"
format_stamp: "Formatted at 2026-06-19 12:24:46 on dist-test-slave-kr9h"
server_key: "8a3f77ba15eaee15b5459d5f53b5ed83"
server_key_iv: "6e6beaafbc746ace188ef811a8939846"
server_key_version: "encryptionkey@0"
I20260619 12:24:46.092550 12291 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260619 12:24:46.095042 12305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:46.096253 12291 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260619 12:24:46.096398 12291 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "6cb122db71724c79b442a06354f89ce6"
format_stamp: "Formatted at 2026-06-19 12:24:46 on dist-test-slave-kr9h"
server_key: "8a3f77ba15eaee15b5459d5f53b5ed83"
server_key_iv: "6e6beaafbc746ace188ef811a8939846"
server_key_version: "encryptionkey@0"
I20260619 12:24:46.096510 12291 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:46.107430 12291 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:46.108089 12291 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:46.108294 12291 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:46.115295 12291 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:34943
I20260619 12:24:46.115301 12357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:34943 every 8 connection(s)
I20260619 12:24:46.116250 12291 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:46.119546 12358 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:46.123992 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 12291
I20260619 12:24:46.124095 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:46.124380 7071 external_mini_cluster.cc:1428] Setting key a0155d903fc0c43f9f6fb775799fc7a9
I20260619 12:24:46.125744 12358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6: Bootstrap starting.
I20260619 12:24:46.128497 12358 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:46.129393 12358 log.cc:824] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:46.131518 12358 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6: No bootstrap required, opened a new log
I20260619 12:24:46.135015 12361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:46.125855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51926 (local address 127.6.231.254:34943)
0619 12:24:46.126567 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:46.126591 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:46.126632 (+ 41us) server_negotiation.cc:407] Connection header received
0619 12:24:46.127492 (+ 860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:46.127515 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:46.128021 (+ 506us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:46.128396 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:46.128888 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.130214 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.131145 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.131835 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.132001 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:46.132786 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:46.132793 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:46.132811 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:46.133323 (+ 512us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:46.133403 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:46.133461 (+ 58us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:46.133561 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:46.133678 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 12:24:46.133949 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":110,"threads_started":1}
I20260619 12:24:46.135982 12358 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb122db71724c79b442a06354f89ce6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34943 } }
I20260619 12:24:46.136191 12358 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:46.136255 12358 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cb122db71724c79b442a06354f89ce6, State: Initialized, Role: FOLLOWER
I20260619 12:24:46.136677 12358 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb122db71724c79b442a06354f89ce6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34943 } }
I20260619 12:24:46.136826 12358 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:46.136916 12358 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:46.137043 12358 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:46.137910 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:24:46.138178 12358 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb122db71724c79b442a06354f89ce6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34943 } }
I20260619 12:24:46.138530 12358 leader_election.cc:302] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cb122db71724c79b442a06354f89ce6; no voters:
I20260619 12:24:46.138892 12358 leader_election.cc:288] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:46.138969 12363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:46.139178 12363 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [term 1 LEADER]: Becoming Leader. State: Replica: 6cb122db71724c79b442a06354f89ce6, State: Running, Role: LEADER
I20260619 12:24:46.139495 12363 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb122db71724c79b442a06354f89ce6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34943 } }
I20260619 12:24:46.140113 12358 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:46.141104 12364 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cb122db71724c79b442a06354f89ce6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb122db71724c79b442a06354f89ce6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34943 } } }
I20260619 12:24:46.141233 12364 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:46.141639 12372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:46.141600 12365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cb122db71724c79b442a06354f89ce6. Latest consensus state: current_term: 1 leader_uuid: "6cb122db71724c79b442a06354f89ce6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cb122db71724c79b442a06354f89ce6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34943 } } }
I20260619 12:24:46.141800 12365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6 [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:46.144234 12372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:46.149827 12372 catalog_manager.cc:1347] Generated new cluster ID: df07864ad8ee46cf95d5264ae7e905ff
I20260619 12:24:46.149897 12372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:46.165699 12372 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:46.166687 12372 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:46.180327 12372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cb122db71724c79b442a06354f89ce6: Generated new TSK 0
I20260619 12:24:46.181079 12372 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260619 12:24:46.211773 12388 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260619 12:24:46.211875 12388 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260619 12:24:46.212601 12361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:46.196725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51934 (local address 127.6.231.254:34943)
0619 12:24:46.196856 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:46.196861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:46.197010 (+ 149us) server_negotiation.cc:407] Connection header received
0619 12:24:46.197200 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:46.197202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:46.197246 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:46.197362 (+ 116us) server_negotiation.cc:226] Negotiated authn=JWT
0619 12:24:46.197625 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.198161 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.199036 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.199328 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.199436 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:46.199805 (+ 369us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 12:24:46.212167 (+ 12362us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35235/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 12:24:46.212460 (+ 293us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51934: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35235/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 12:24:46.212646 12387 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:24:46.196641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34943 (local address 127.0.0.1:51934)
0619 12:24:46.196926 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:46.196938 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:46.197060 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:46.197360 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:46.197363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:46.197371 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0619 12:24:46.197509 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:46.197522 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.198280 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.198284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:46.198916 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:46.198923 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.199442 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.199446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:46.199537 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:46.199568 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 12:24:46.212370 (+ 12802us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35235/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0619 12:24:46.212514 (+ 144us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.231.254:34943: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35235/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":97,"threads_started":1}
W20260619 12:24:46.212674 12361 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51934: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35235/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260619 12:24:46.213346 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 12291
2026-06-19T12:24:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (314 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260619 12:24:46.251749 7071 mini_oidc.cc:152] Starting JWKS server
I20260619 12:24:46.252614 7071 webserver.cc:460] Webserver started at https://127.0.0.1:44929/ using document root <none> and password file <none>
I20260619 12:24:46.252723 7071 mini_oidc.cc:189] Starting OIDC Discovery server
I20260619 12:24:46.253023 7071 webserver.cc:460] Webserver started at http://127.0.0.1:42745/ using document root <none> and password file <none>
2026-06-19T12:24:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:46Z Disabled control of system clock
I20260619 12:24:46.261360 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:38539
--webserver_interface=127.6.231.254
--webserver_port=0
--builtin_ntp_servers=127.6.231.212:40853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:38539
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44929/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 12:24:46.369478 12394 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:46.369782 12394 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:46.369858 12394 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:46.373907 12394 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:46.373996 12394 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:46.374022 12394 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:46.374063 12394 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:46.374094 12394 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 12:24:46.374130 12394 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 12:24:46.374161 12394 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44929/jwks.json
I20260619 12:24:46.378927 12394 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:40853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:38539
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:38539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44929/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12394
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:46.380030 12394 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:46.381469 12394 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:46.389834 12399 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:46.389979 12394 server_base.cc:1034] running on GCE node
W20260619 12:24:46.389850 12402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:46.389926 12400 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:46.390632 12394 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:46.391737 12394 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:46.392894 12394 hybrid_clock.cc:648] HybridClock initialized: now 1781871886392874 us; error 30 us; skew 500 ppm
I20260619 12:24:46.394721 12394 webserver.cc:460] Webserver started at http://127.6.231.254:42539/ using document root <none> and password file <none>
I20260619 12:24:46.395241 12394 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:46.395299 12394 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:46.395464 12394 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:46.397697 12394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "d18d3ce186d54d2b8360ab622f47dba1"
format_stamp: "Formatted at 2026-06-19 12:24:46 on dist-test-slave-kr9h"
server_key: "eec93b4d36177f95b6aea09f99744dc6"
server_key_iv: "6278e5f250942c7e0563c353f5cd842f"
server_key_version: "encryptionkey@0"
I20260619 12:24:46.398247 12394 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "d18d3ce186d54d2b8360ab622f47dba1"
format_stamp: "Formatted at 2026-06-19 12:24:46 on dist-test-slave-kr9h"
server_key: "eec93b4d36177f95b6aea09f99744dc6"
server_key_iv: "6278e5f250942c7e0563c353f5cd842f"
server_key_version: "encryptionkey@0"
I20260619 12:24:46.401818 12394 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260619 12:24:46.404168 12408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:46.405354 12394 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:24:46.405514 12394 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "d18d3ce186d54d2b8360ab622f47dba1"
format_stamp: "Formatted at 2026-06-19 12:24:46 on dist-test-slave-kr9h"
server_key: "eec93b4d36177f95b6aea09f99744dc6"
server_key_iv: "6278e5f250942c7e0563c353f5cd842f"
server_key_version: "encryptionkey@0"
I20260619 12:24:46.405611 12394 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:46.423118 12394 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:46.423755 12394 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:46.423907 12394 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:46.430630 12394 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:38539
I20260619 12:24:46.430640 12460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:38539 every 8 connection(s)
I20260619 12:24:46.431577 12394 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:46.434648 12461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:46.436951 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 12394
I20260619 12:24:46.437055 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:46.437361 7071 external_mini_cluster.cc:1428] Setting key c4e311671c3d55bf9c848ab5b35e67ec
I20260619 12:24:46.440289 12461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1: Bootstrap starting.
I20260619 12:24:46.442719 12461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:46.443583 12461 log.cc:824] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:46.445716 12461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1: No bootstrap required, opened a new log
I20260619 12:24:46.447230 12464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:46.438824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43152 (local address 127.6.231.254:38539)
0619 12:24:46.439445 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:46.439459 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:46.439493 (+ 34us) server_negotiation.cc:407] Connection header received
0619 12:24:46.440089 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:46.440108 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:46.440549 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:46.440827 (+ 278us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:46.441162 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.442246 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.443475 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.444022 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.444146 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:46.445058 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:46.445066 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:46.445082 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:46.445656 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:46.445735 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:46.445765 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:46.445898 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:46.446031 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 12:24:46.446316 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":134,"threads_started":1}
I20260619 12:24:46.450248 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:24:46.450248 12461 raft_consensus.cc:348] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18d3ce186d54d2b8360ab622f47dba1" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 38539 } }
I20260619 12:24:46.450508 12461 raft_consensus.cc:374] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:46.450582 12461 raft_consensus.cc:729] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d18d3ce186d54d2b8360ab622f47dba1, State: Initialized, Role: FOLLOWER
I20260619 12:24:46.451035 12461 consensus_queue.cc:260] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18d3ce186d54d2b8360ab622f47dba1" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 38539 } }
I20260619 12:24:46.451186 12461 raft_consensus.cc:388] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:46.451251 12461 raft_consensus.cc:482] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:46.451345 12461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:46.452417 12461 raft_consensus.cc:504] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18d3ce186d54d2b8360ab622f47dba1" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 38539 } }
I20260619 12:24:46.452884 12461 leader_election.cc:302] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d18d3ce186d54d2b8360ab622f47dba1; no voters:
I20260619 12:24:46.453285 12461 leader_election.cc:288] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:46.453409 12466 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:46.453696 12466 raft_consensus.cc:686] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [term 1 LEADER]: Becoming Leader. State: Replica: d18d3ce186d54d2b8360ab622f47dba1, State: Running, Role: LEADER
I20260619 12:24:46.454021 12466 consensus_queue.cc:237] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18d3ce186d54d2b8360ab622f47dba1" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 38539 } }
I20260619 12:24:46.454773 12461 sys_catalog.cc:564] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:46.455698 12468 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d18d3ce186d54d2b8360ab622f47dba1. Latest consensus state: current_term: 1 leader_uuid: "d18d3ce186d54d2b8360ab622f47dba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18d3ce186d54d2b8360ab622f47dba1" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 38539 } } }
I20260619 12:24:46.455822 12468 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:46.456053 12467 sys_catalog.cc:455] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d18d3ce186d54d2b8360ab622f47dba1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d18d3ce186d54d2b8360ab622f47dba1" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 38539 } } }
I20260619 12:24:46.456138 12467 sys_catalog.cc:458] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1 [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:46.456869 12475 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:46.459429 12475 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:46.464808 12475 catalog_manager.cc:1347] Generated new cluster ID: 697ecbb1a57a4518902d98eb3d0c5713
I20260619 12:24:46.464882 12475 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:46.495954 12475 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:46.496938 12475 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:46.501312 12475 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d18d3ce186d54d2b8360ab622f47dba1: Generated new TSK 0
I20260619 12:24:46.502043 12475 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:24:46.519980 12464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:46.514462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43162 (local address 127.6.231.254:38539)
0619 12:24:46.514588 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:46.514592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:46.514729 (+ 137us) server_negotiation.cc:407] Connection header received
0619 12:24:46.514870 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:46.514873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:46.514934 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:46.515055 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0619 12:24:46.515262 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.516197 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.517113 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.517535 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.517647 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:46.517950 (+ 303us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 12:24:46.519571 (+ 1621us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44929/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 12:24:46.519829 (+ 258us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:43162: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44929/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 12:24:46.520062 12464 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:43162: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44929/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260619 12:24:46.521548 12490 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:24:46.514368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:38539 (local address 127.0.0.1:43162)
0619 12:24:46.514629 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:46.514641 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:46.514737 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:46.515062 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:46.515064 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:46.515070 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0619 12:24:46.515156 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:46.515160 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.516336 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.516340 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:46.516989 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:46.516998 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.517651 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.517654 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:46.517715 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:46.517736 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 12:24:46.519738 (+ 2002us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44929/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0619 12:24:46.521356 (+ 1618us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.231.254:38539: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44929/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":1413,"thread_start_us":101,"threads_started":1}
I20260619 12:24:46.522130 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 12394
E20260619 12:24:46.531015 12491 webserver.cc:533] Webserver: sslize failed:
E20260619 12:24:46.531090 12491 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-06-19T12:24:46Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (309 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-06-19T12:24:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:46Z Disabled control of system clock
I20260619 12:24:46.570169 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:34169
--webserver_interface=127.6.231.254
--webserver_port=0
--builtin_ntp_servers=127.6.231.212:42233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:34169
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/testchainca.cert with env {}
W20260619 12:24:46.682617 12495 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:46.682902 12495 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:46.682943 12495 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:46.687336 12495 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:46.687402 12495 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:46.687418 12495 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:46.687436 12495 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:46.687451 12495 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260619 12:24:46.687505 12495 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260619 12:24:46.687532 12495 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260619 12:24:46.693207 12495 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:42233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:34169
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:34169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12495
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:46.694355 12495 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:46.695459 12495 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:46.704355 12503 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:46.704370 12501 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:46.704372 12500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:46.704814 12495 server_base.cc:1034] running on GCE node
I20260619 12:24:46.705699 12495 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:46.706959 12495 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:46.708146 12495 hybrid_clock.cc:648] HybridClock initialized: now 1781871886708119 us; error 44 us; skew 500 ppm
I20260619 12:24:46.710228 12495 webserver.cc:460] Webserver started at http://127.6.231.254:41221/ using document root <none> and password file <none>
I20260619 12:24:46.710824 12495 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:46.710886 12495 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:46.711097 12495 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:46.713158 12495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "d1bd895940c64175960216e1dec9a6ac"
format_stamp: "Formatted at 2026-06-19 12:24:46 on dist-test-slave-kr9h"
server_key: "6eb39dae9dd7f8e65221e04411392ef0"
server_key_iv: "25bf80b3da6d42531990c9c0fb4d65ac"
server_key_version: "encryptionkey@0"
I20260619 12:24:46.713783 12495 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "d1bd895940c64175960216e1dec9a6ac"
format_stamp: "Formatted at 2026-06-19 12:24:46 on dist-test-slave-kr9h"
server_key: "6eb39dae9dd7f8e65221e04411392ef0"
server_key_iv: "25bf80b3da6d42531990c9c0fb4d65ac"
server_key_version: "encryptionkey@0"
I20260619 12:24:46.717566 12495 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 12:24:46.720275 12509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:46.721603 12495 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:46.721829 12495 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "d1bd895940c64175960216e1dec9a6ac"
format_stamp: "Formatted at 2026-06-19 12:24:46 on dist-test-slave-kr9h"
server_key: "6eb39dae9dd7f8e65221e04411392ef0"
server_key_iv: "25bf80b3da6d42531990c9c0fb4d65ac"
server_key_version: "encryptionkey@0"
I20260619 12:24:46.721992 12495 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:46.737280 12495 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:46.738070 12495 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:46.738411 12495 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:46.745854 12495 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:34169
I20260619 12:24:46.745865 12561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:34169 every 8 connection(s)
I20260619 12:24:46.746906 12495 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:46.750025 12562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:46.755486 12562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac: Bootstrap starting.
I20260619 12:24:46.756278 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 12495
I20260619 12:24:46.756374 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:46.756836 7071 external_mini_cluster.cc:1428] Setting key 4499b784b7fdd2cc780bca6e3b1304da
I20260619 12:24:46.758309 12562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:46.759060 12562 log.cc:824] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:46.761164 12562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac: No bootstrap required, opened a new log
I20260619 12:24:46.765674 12562 raft_consensus.cc:348] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bd895940c64175960216e1dec9a6ac" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34169 } }
I20260619 12:24:46.765923 12562 raft_consensus.cc:374] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:46.766000 12562 raft_consensus.cc:729] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1bd895940c64175960216e1dec9a6ac, State: Initialized, Role: FOLLOWER
I20260619 12:24:46.766431 12565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:46.758110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54874 (local address 127.6.231.254:34169)
0619 12:24:46.758806 (+ 696us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:46.758820 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:46.758857 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:24:46.759560 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:46.759578 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:46.759927 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:46.760207 (+ 280us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:46.760547 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.761617 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.762858 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.763328 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.763445 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:46.764302 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:46.764307 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:46.764319 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:24:46.764716 (+ 397us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:46.764841 (+ 125us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:24:46.764870 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:46.764977 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:46.765110 (+ 133us) server_negotiation.cc:299] Negotiation successful
0619 12:24:46.765407 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":158,"threads_started":1}
I20260619 12:24:46.766445 12562 consensus_queue.cc:260] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bd895940c64175960216e1dec9a6ac" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34169 } }
I20260619 12:24:46.766695 12562 raft_consensus.cc:388] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:46.766757 12562 raft_consensus.cc:482] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:46.766844 12562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:46.767856 12562 raft_consensus.cc:504] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bd895940c64175960216e1dec9a6ac" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34169 } }
I20260619 12:24:46.768224 12562 leader_election.cc:302] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d1bd895940c64175960216e1dec9a6ac; no voters:
I20260619 12:24:46.768558 12562 leader_election.cc:288] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:46.768671 12567 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:46.768915 12567 raft_consensus.cc:686] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [term 1 LEADER]: Becoming Leader. State: Replica: d1bd895940c64175960216e1dec9a6ac, State: Running, Role: LEADER
I20260619 12:24:46.769526 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:24:46.769644 12567 consensus_queue.cc:237] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bd895940c64175960216e1dec9a6ac" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34169 } }
I20260619 12:24:46.769886 12562 sys_catalog.cc:564] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:46.771346 12569 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader d1bd895940c64175960216e1dec9a6ac. Latest consensus state: current_term: 1 leader_uuid: "d1bd895940c64175960216e1dec9a6ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bd895940c64175960216e1dec9a6ac" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34169 } } }
I20260619 12:24:46.771482 12569 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:46.771487 12568 sys_catalog.cc:455] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d1bd895940c64175960216e1dec9a6ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d1bd895940c64175960216e1dec9a6ac" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34169 } } }
I20260619 12:24:46.771852 12568 sys_catalog.cc:458] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac [sys.catalog]: This master's current role is: LEADER
W20260619 12:24:46.774514 12582 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:24:46.774578 12582 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:24:46.774662 12579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:46.775313 12579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:46.780972 12579 catalog_manager.cc:1347] Generated new cluster ID: d752402e33314803a425145cc507199a
I20260619 12:24:46.781044 12579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:46.802310 12579 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:46.803283 12579 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:46.813261 12579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d1bd895940c64175960216e1dec9a6ac: Generated new TSK 0
I20260619 12:24:46.814178 12579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:24:47.281281 12565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:46.826367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54884 (local address 127.6.231.254:34169)
0619 12:24:46.826501 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:46.826506 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:46.826699 (+ 193us) server_negotiation.cc:407] Connection header received
0619 12:24:46.826847 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:46.826850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:46.826908 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:46.827030 (+ 122us) server_negotiation.cc:226] Negotiated authn=JWT
0619 12:24:46.827312 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.828144 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.829012 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.829356 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.829501 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:46.829867 (+ 366us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0619 12:24:47.280887 (+451020us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 12:24:47.281160 (+ 273us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:54884: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":47}
W20260619 12:24:47.281351 12565 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:54884: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260619 12:24:47.281567 12591 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:24:46.826281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34169 (local address 127.0.0.1:54884)
0619 12:24:46.826603 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:46.826616 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:46.826717 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:46.827058 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:46.827064 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:46.827074 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0619 12:24:46.827178 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:46.827183 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.828276 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.828279 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:46.828904 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:46.828911 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:46.829578 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:46.829582 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:46.829653 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:46.829683 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0619 12:24:47.281130 (+451447us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0619 12:24:47.281299 (+ 169us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.6.231.254:34169: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":103,"threads_started":1}
I20260619 12:24:47.282244 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 12495
2026-06-19T12:24:47Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (739 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (122 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (125 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:24:47 dist-test-slave-kr9h krb5kdc[12603](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:24:47 dist-test-slave-kr9h krb5kdc[12603](info): set up 2 sockets
Jun 19 12:24:47 dist-test-slave-kr9h krb5kdc[12603](info): commencing operation
krb5kdc: starting...
W20260619 12:24:48.581861 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.032s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:24:48 dist-test-slave-kr9h krb5kdc[12603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:24:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:24:48.700811 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:37939
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:37939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:48.812016 12616 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:48.812304 12616 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:48.812345 12616 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:48.816360 12616 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:48.816427 12616 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:48.816447 12616 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:48.816464 12616 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:48.816479 12616 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:48.821794 12616 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:37939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:37939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12616
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:48.822835 12616 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:48.823829 12616 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:48.832489 12622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:48.832552 12616 server_base.cc:1034] running on GCE node
W20260619 12:24:48.832489 12624 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:48.832516 12621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:48.833352 12616 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:48.834633 12616 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:48.835860 12616 hybrid_clock.cc:648] HybridClock initialized: now 1781871888835824 us; error 51 us; skew 500 ppm
Jun 19 12:24:48 dist-test-slave-kr9h krb5kdc[12603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871888, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:48.838970 12616 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:24:48.839870 12616 webserver.cc:460] Webserver started at http://127.6.231.254:39057/ using document root <none> and password file <none>
I20260619 12:24:48.840438 12616 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:48.840498 12616 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:48.840663 12616 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:48.842832 12616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "5958a06bdd92441e964280840b88e4de"
format_stamp: "Formatted at 2026-06-19 12:24:48 on dist-test-slave-kr9h"
server_key: "0dfbffa6d900679141624c5cd01c38f8"
server_key_iv: "9498f41f43c790361ced7f91a5267fc7"
server_key_version: "encryptionkey@0"
I20260619 12:24:48.843366 12616 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "5958a06bdd92441e964280840b88e4de"
format_stamp: "Formatted at 2026-06-19 12:24:48 on dist-test-slave-kr9h"
server_key: "0dfbffa6d900679141624c5cd01c38f8"
server_key_iv: "9498f41f43c790361ced7f91a5267fc7"
server_key_version: "encryptionkey@0"
I20260619 12:24:48.847237 12616 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:24:48.850107 12631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:48.851465 12616 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:48.851636 12616 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "5958a06bdd92441e964280840b88e4de"
format_stamp: "Formatted at 2026-06-19 12:24:48 on dist-test-slave-kr9h"
server_key: "0dfbffa6d900679141624c5cd01c38f8"
server_key_iv: "9498f41f43c790361ced7f91a5267fc7"
server_key_version: "encryptionkey@0"
I20260619 12:24:48.851795 12616 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:48.873672 12616 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:48.877234 12616 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:48.877460 12616 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:48.885054 12616 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:37939
I20260619 12:24:48.885344 12683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:37939 every 8 connection(s)
I20260619 12:24:48.886263 12616 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:48.887398 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 12616
I20260619 12:24:48.887496 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:48.887874 7071 external_mini_cluster.cc:1428] Setting key 27d1d58cf32a4dbb6b486676fa3612d2
I20260619 12:24:48.889496 12684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:48.896466 12684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de: Bootstrap starting.
Jun 19 12:24:48 dist-test-slave-kr9h krb5kdc[12603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871888, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:48.898978 12684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:48.899935 12684 log.cc:824] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:48.902221 12684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de: No bootstrap required, opened a new log
I20260619 12:24:48.904174 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:48.889490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55534 (local address 127.6.231.254:37939)
0619 12:24:48.890063 (+ 573us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:48.890082 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:48.890124 (+ 42us) server_negotiation.cc:407] Connection header received
0619 12:24:48.890971 (+ 847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:48.890993 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:48.891417 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:48.891792 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:48.892536 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:48.894019 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:48.895069 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:48.895800 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:48.895961 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:48.898587 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:48.898593 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:48.898609 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:48.898641 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:48.900724 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:48.901164 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:48.901169 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:48.901175 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:48.901306 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:48.901690 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:48.901693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:48.901695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:48.902191 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:48.902377 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:48.902708 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 12:24:48.902999 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":426,"thread_start_us":140,"threads_started":1}
I20260619 12:24:48.906096 12684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5958a06bdd92441e964280840b88e4de" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37939 } }
I20260619 12:24:48.906345 12684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:48.906445 12684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5958a06bdd92441e964280840b88e4de, State: Initialized, Role: FOLLOWER
I20260619 12:24:48.906893 12684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5958a06bdd92441e964280840b88e4de" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37939 } }
I20260619 12:24:48.907011 12684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:48.907052 12684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:48.907171 12684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:48.908123 12684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5958a06bdd92441e964280840b88e4de" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37939 } }
I20260619 12:24:48.908478 12684 leader_election.cc:302] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5958a06bdd92441e964280840b88e4de; no voters:
I20260619 12:24:48.908794 12684 leader_election.cc:288] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:48.908949 12689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:48.909204 12689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [term 1 LEADER]: Becoming Leader. State: Replica: 5958a06bdd92441e964280840b88e4de, State: Running, Role: LEADER
I20260619 12:24:48.909662 12689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5958a06bdd92441e964280840b88e4de" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37939 } }
I20260619 12:24:48.910109 12684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:48.911289 12690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5958a06bdd92441e964280840b88e4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5958a06bdd92441e964280840b88e4de" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37939 } } }
I20260619 12:24:48.911428 12690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:48.911705 12691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5958a06bdd92441e964280840b88e4de. Latest consensus state: current_term: 1 leader_uuid: "5958a06bdd92441e964280840b88e4de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5958a06bdd92441e964280840b88e4de" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37939 } } }
I20260619 12:24:48.911777 12691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:48.911804 12698 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:48.915292 12698 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:48.921638 12698 catalog_manager.cc:1347] Generated new cluster ID: dc9169217fd146f7adbd9395a50538ff
I20260619 12:24:48.921731 12698 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:48.937474 12698 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:48.938582 12698 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:48.943926 12698 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5958a06bdd92441e964280840b88e4de: Generated new TSK 0
I20260619 12:24:48.944669 12698 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:24:48.982918 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:37939
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:49.103937 12710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:49.104349 12710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:49.104439 12710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:49.108628 12710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:49.108697 12710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:49.108763 12710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:24:49.114501 12710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:37939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:49.115545 12710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:49.116513 12710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:24:49.126106 12710 server_base.cc:1034] running on GCE node
W20260619 12:24:49.126060 12715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:49.126089 12716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:49.126070 12718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:49.126713 12710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:49.127347 12710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:49.128562 12710 hybrid_clock.cc:648] HybridClock initialized: now 1781871889128546 us; error 29 us; skew 500 ppm
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:49.131680 12710 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:24:49.132542 12710 webserver.cc:460] Webserver started at http://127.6.231.193:34447/ using document root <none> and password file <none>
I20260619 12:24:49.133100 12710 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:49.133172 12710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:49.133461 12710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:49.135588 12710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "eb07db411cb04ddaa19b395c4ed426d3"
format_stamp: "Formatted at 2026-06-19 12:24:49 on dist-test-slave-kr9h"
server_key: "20498eb50d99adc7df7139aa24cbcbc0"
server_key_iv: "21f39b4e622ae7ca37501ef87c086ac9"
server_key_version: "encryptionkey@0"
I20260619 12:24:49.136173 12710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "eb07db411cb04ddaa19b395c4ed426d3"
format_stamp: "Formatted at 2026-06-19 12:24:49 on dist-test-slave-kr9h"
server_key: "20498eb50d99adc7df7139aa24cbcbc0"
server_key_iv: "21f39b4e622ae7ca37501ef87c086ac9"
server_key_version: "encryptionkey@0"
I20260619 12:24:49.139946 12710 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260619 12:24:49.142421 12725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:49.143676 12710 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:24:49.143812 12710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "eb07db411cb04ddaa19b395c4ed426d3"
format_stamp: "Formatted at 2026-06-19 12:24:49 on dist-test-slave-kr9h"
server_key: "20498eb50d99adc7df7139aa24cbcbc0"
server_key_iv: "21f39b4e622ae7ca37501ef87c086ac9"
server_key_version: "encryptionkey@0"
I20260619 12:24:49.143923 12710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:49.158731 12710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:49.162014 12710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:49.162240 12710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:49.162837 12710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:49.163658 12710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:49.163730 12710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:49.163800 12710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:49.163846 12710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:49.172777 12710 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:36405
I20260619 12:24:49.172801 12838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:36405 every 8 connection(s)
I20260619 12:24:49.173705 12710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:24:49.180166 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 12710
I20260619 12:24:49.180310 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:24:49.180611 7071 external_mini_cluster.cc:1428] Setting key 0a63a49f27b387edf55b13800ee1e1ea
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:49.188417 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:49.175466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:35143 (local address 127.6.231.254:37939)
0619 12:24:49.175611 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:49.175617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:49.176368 (+ 751us) server_negotiation.cc:407] Connection header received
0619 12:24:49.177796 (+ 1428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:49.177799 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:49.177860 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:49.177960 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:49.179103 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.179640 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.180930 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.181263 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.181378 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:49.184525 (+ 3147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:49.184529 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:49.184532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:49.184564 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:49.186151 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:49.186691 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:49.186696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:49.186698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:49.186784 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:49.187129 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:49.187134 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:49.187136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:49.187329 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:49.187433 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:49.188099 (+ 666us) server_negotiation.cc:299] Negotiation successful
0619 12:24:49.188239 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:24:49.189296 12841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:49.175785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:37939 (local address 127.6.231.193:35143)
0619 12:24:49.176218 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:49.176258 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:49.177582 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:49.178129 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:49.178136 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:49.178485 (+ 349us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:49.178951 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:49.178964 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.179764 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.179768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:49.180812 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:49.180821 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.181415 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.181418 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:49.181624 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:49.182311 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:49.182333 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:49.184337 (+ 2004us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:49.186277 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:49.186284 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:49.186301 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:49.186584 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:49.186885 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:49.186888 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:49.186890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:49.187035 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:49.187443 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:49.187460 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:49.187759 (+ 299us) client_negotiation.cc:769] Sending connection context
0619 12:24:49.188060 (+ 301us) client_negotiation.cc:240] Negotiation successful
0619 12:24:49.188410 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":111,"threads_started":1}
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
I20260619 12:24:49.190724 12839 heartbeater.cc:347] Connected to a master server at 127.6.231.254:37939
I20260619 12:24:49.190933 12839 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:49.191427 12839 heartbeater.cc:511] Master 127.6.231.254:37939 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:24:49.193113 12646 ts_manager.cc:194] Registered new tserver with Master: eb07db411cb04ddaa19b395c4ed426d3 (127.6.231.193:36405)
I20260619 12:24:49.194561 12646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:35143
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:24:49.208874 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:37939
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:49.324198 12844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:49.324640 12844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:49.324729 12844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:49.329108 12844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:49.329223 12844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:49.329324 12844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:24:49.337121 12844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:37939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:49.338411 12844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:49.339552 12844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:49.349655 12850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:49.349761 12849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:49.349666 12852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:49.349990 12844 server_base.cc:1034] running on GCE node
I20260619 12:24:49.350807 12844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:49.351541 12844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:49.352798 12844 hybrid_clock.cc:648] HybridClock initialized: now 1781871889352752 us; error 63 us; skew 500 ppm
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:49.356079 12844 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:24:49.357156 12844 webserver.cc:460] Webserver started at http://127.6.231.194:40539/ using document root <none> and password file <none>
I20260619 12:24:49.357795 12844 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:49.357859 12844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:49.358078 12844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:49.360196 12844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "b8575f909735403384da9be2642a207d"
format_stamp: "Formatted at 2026-06-19 12:24:49 on dist-test-slave-kr9h"
server_key: "8382e990dafff2d7c1d755eedfcfdaa0"
server_key_iv: "37d89b273102361e1e6de064a59de9ba"
server_key_version: "encryptionkey@0"
I20260619 12:24:49.360781 12844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "b8575f909735403384da9be2642a207d"
format_stamp: "Formatted at 2026-06-19 12:24:49 on dist-test-slave-kr9h"
server_key: "8382e990dafff2d7c1d755eedfcfdaa0"
server_key_iv: "37d89b273102361e1e6de064a59de9ba"
server_key_version: "encryptionkey@0"
I20260619 12:24:49.364789 12844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260619 12:24:49.367688 12859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:49.369068 12844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:24:49.369181 12844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "b8575f909735403384da9be2642a207d"
format_stamp: "Formatted at 2026-06-19 12:24:49 on dist-test-slave-kr9h"
server_key: "8382e990dafff2d7c1d755eedfcfdaa0"
server_key_iv: "37d89b273102361e1e6de064a59de9ba"
server_key_version: "encryptionkey@0"
I20260619 12:24:49.369334 12844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:49.393769 12844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:49.397362 12844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:49.397639 12844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:49.398274 12844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:49.399113 12844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:49.399159 12844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:49.399199 12844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:49.399251 12844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:49.408834 12844 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:41283
I20260619 12:24:49.408857 12972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:41283 every 8 connection(s)
I20260619 12:24:49.409965 12844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:24:49.416184 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 12844
I20260619 12:24:49.416308 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:24:49.416626 7071 external_mini_cluster.cc:1428] Setting key a9a8c3baf0d5d8fdebfd7fc4f5e5f08a
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:49.426079 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:49.412224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:33719 (local address 127.6.231.254:37939)
0619 12:24:49.412414 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:49.412419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:49.413331 (+ 912us) server_negotiation.cc:407] Connection header received
0619 12:24:49.414839 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:49.414842 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:49.414892 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:49.414984 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:49.416473 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.417152 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.418772 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.418989 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.419090 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:49.421978 (+ 2888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:49.421980 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:49.421982 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:49.422005 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:49.423654 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:49.424346 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:49.424349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:49.424350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:49.424425 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:49.424845 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:49.424847 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:49.424849 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:49.425030 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:49.425116 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:49.425838 (+ 722us) server_negotiation.cc:299] Negotiation successful
0619 12:24:49.425953 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
I20260619 12:24:49.427273 12975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:49.412683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:37939 (local address 127.6.231.194:33719)
0619 12:24:49.413164 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:49.413212 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:49.414598 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:49.415159 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:49.415167 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:49.415540 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:49.416250 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:49.416273 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.417297 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.417300 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:49.418641 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:49.418655 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.419166 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.419173 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:49.419406 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:49.420141 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:49.420158 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:49.421774 (+ 1616us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:49.423783 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:49.423791 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:49.423813 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:49.424228 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:49.424520 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:49.424525 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:49.424530 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:49.424725 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:49.425144 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:49.425152 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:49.425525 (+ 373us) client_negotiation.cc:769] Sending connection context
0619 12:24:49.425746 (+ 221us) client_negotiation.cc:240] Negotiation successful
0619 12:24:49.426146 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":117,"threads_started":1}
I20260619 12:24:49.428500 12973 heartbeater.cc:347] Connected to a master server at 127.6.231.254:37939
I20260619 12:24:49.428786 12973 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:49.429347 12973 heartbeater.cc:511] Master 127.6.231.254:37939 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
I20260619 12:24:49.430542 12646 ts_manager.cc:194] Registered new tserver with Master: b8575f909735403384da9be2642a207d (127.6.231.194:41283)
I20260619 12:24:49.431144 12646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:33719
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:24:49.444841 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:37939
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:49.557591 12978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:49.557870 12978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:49.557914 12978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:49.562016 12978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:49.562083 12978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:49.562141 12978 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:24:49.567477 12978 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:37939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:49.568560 12978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:49.569655 12978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:49.579455 12986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:49.579455 12983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:49.579543 12984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:49.579859 12978 server_base.cc:1034] running on GCE node
I20260619 12:24:49.580278 12978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:49.580935 12978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:49.582182 12978 hybrid_clock.cc:648] HybridClock initialized: now 1781871889582093 us; error 67 us; skew 500 ppm
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:49.585059 12978 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:24:49.586040 12978 webserver.cc:460] Webserver started at http://127.6.231.195:42967/ using document root <none> and password file <none>
I20260619 12:24:49.586619 12978 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:49.586709 12978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:49.586911 12978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:49.588948 12978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "ecfdfa0e5be54794ab48034d1d263ac2"
format_stamp: "Formatted at 2026-06-19 12:24:49 on dist-test-slave-kr9h"
server_key: "d11e7aade543b7dd6bb15fe721270f15"
server_key_iv: "82699dd783cebaf06a54824e65c3c2ca"
server_key_version: "encryptionkey@0"
I20260619 12:24:49.589574 12978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "ecfdfa0e5be54794ab48034d1d263ac2"
format_stamp: "Formatted at 2026-06-19 12:24:49 on dist-test-slave-kr9h"
server_key: "d11e7aade543b7dd6bb15fe721270f15"
server_key_iv: "82699dd783cebaf06a54824e65c3c2ca"
server_key_version: "encryptionkey@0"
I20260619 12:24:49.593309 12978 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260619 12:24:49.595918 12993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:49.597129 12978 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:24:49.597270 12978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "ecfdfa0e5be54794ab48034d1d263ac2"
format_stamp: "Formatted at 2026-06-19 12:24:49 on dist-test-slave-kr9h"
server_key: "d11e7aade543b7dd6bb15fe721270f15"
server_key_iv: "82699dd783cebaf06a54824e65c3c2ca"
server_key_version: "encryptionkey@0"
I20260619 12:24:49.597386 12978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:49.620190 12978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:49.624025 12978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:49.624284 12978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:49.624969 12978 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:49.625921 12978 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:49.626001 12978 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:49.626088 12978 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:49.626130 12978 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:49.636171 12978 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:34485
I20260619 12:24:49.636193 13106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:34485 every 8 connection(s)
I20260619 12:24:49.637131 12978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:24:49.641958 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 12978
I20260619 12:24:49.642122 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:24:49.642419 7071 external_mini_cluster.cc:1428] Setting key fb345087cf699df7419b75cd0b0d253f
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:24:49.651784 12687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:49.639135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:53287 (local address 127.6.231.254:37939)
0619 12:24:49.639284 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:49.639288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:49.640183 (+ 895us) server_negotiation.cc:407] Connection header received
0619 12:24:49.641623 (+ 1440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:49.641626 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:49.641687 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:49.641795 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:49.643303 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.644074 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.645372 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.645695 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.645781 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:49.648404 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:49.648407 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:49.648409 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:49.648433 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:49.649960 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:49.650413 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:49.650415 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:49.650417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:49.650537 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:49.650826 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:49.650828 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:49.650829 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:49.650998 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:49.651065 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:49.651558 (+ 493us) server_negotiation.cc:299] Negotiation successful
0619 12:24:49.651656 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:24:49.652629 13109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:49.639564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:37939 (local address 127.6.231.195:53287)
0619 12:24:49.640023 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:49.640066 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:49.641337 (+ 1271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:49.641982 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:49.641990 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:49.642428 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:49.643148 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:49.643168 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.644200 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.644203 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:49.645250 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:49.645259 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.645832 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.645836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:49.646007 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:49.646709 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:49.646724 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:49.648245 (+ 1521us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:49.650061 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:49.650066 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:49.650078 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:49.650329 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:49.650625 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:49.650627 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:49.650628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:49.650752 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:49.651097 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:49.651102 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:49.651326 (+ 224us) client_negotiation.cc:769] Sending connection context
0619 12:24:49.651479 (+ 153us) client_negotiation.cc:240] Negotiation successful
0619 12:24:49.651737 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":116,"threads_started":1}
I20260619 12:24:49.653990 13107 heartbeater.cc:347] Connected to a master server at 127.6.231.254:37939
I20260619 12:24:49.654253 13107 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:49.654770 13107 heartbeater.cc:511] Master 127.6.231.254:37939 requested a full tablet report, sending...
I20260619 12:24:49.656037 12646 ts_manager.cc:194] Registered new tserver with Master: ecfdfa0e5be54794ab48034d1d263ac2 (127.6.231.195:34485)
I20260619 12:24:49.656657 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:24:49.656808 12646 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:53287
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/test-admin.keytab.
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:49.674242 7071 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260619 12:24:49.677366 13114 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:24:49.675773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58314 (local address 127.6.231.193:36405)
0619 12:24:49.676126 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:49.676140 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:49.676191 (+ 51us) server_negotiation.cc:407] Connection header received
0619 12:24:49.676243 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:49.676249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:49.676453 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:49.676578 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:49.677155 (+ 577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58314: BlockingRecv error: recv got EOF from 127.0.0.1:58314 (error 108)
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":120,"threads_started":1}
W20260619 12:24:49.677345 7071 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/krb5cc' of type FILE
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:49.678483 7071 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:49.679443 7071 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
I20260619 12:24:49.689056 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:49.680359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58328 (local address 127.6.231.193:36405)
0619 12:24:49.680503 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:49.680506 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:49.680521 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:24:49.680634 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:49.680636 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:49.680677 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:49.680746 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:49.681264 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.682227 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.683067 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.683446 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.683543 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:49.685882 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:49.685891 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:49.685904 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:49.685927 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:49.687447 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:49.687900 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:49.687906 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:49.687907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:49.687994 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:49.688220 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:49.688222 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:49.688223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:49.688445 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:49.688595 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:49.688788 (+ 193us) server_negotiation.cc:299] Negotiation successful
0619 12:24:49.688910 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260619 12:24:49.693212 13114 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:24:49.692199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58336 (local address 127.6.231.193:36405)
0619 12:24:49.692329 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:49.692333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:49.692443 (+ 110us) server_negotiation.cc:407] Connection header received
0619 12:24:49.692594 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:49.692597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:49.692640 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:49.692722 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:49.693114 (+ 392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58336: BlockingRecv error: recv got EOF from 127.0.0.1:58336 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260619 12:24:49.693364 7071 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/krb5cc' of type FILE
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:49.694463 7071 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:49.695446 7071 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
I20260619 12:24:49.704582 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:49.696316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58338 (local address 127.6.231.193:36405)
0619 12:24:49.696484 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:49.696487 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:49.696550 (+ 63us) server_negotiation.cc:407] Connection header received
0619 12:24:49.696665 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:49.696667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:49.696700 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:49.696765 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:49.697258 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.697886 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.698699 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.698909 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.698987 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:49.701365 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:49.701368 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:49.701376 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:49.701409 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:49.702777 (+ 1368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:49.703196 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:49.703199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:49.703201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:49.703279 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:49.703588 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:49.703590 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:49.703592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:49.703757 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:49.703862 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:49.704195 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 12:24:49.704372 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
W20260619 12:24:49.707620 13114 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:24:49.706656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.6.231.193:36405)
0619 12:24:49.706790 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:49.706793 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:49.706836 (+ 43us) server_negotiation.cc:407] Connection header received
0619 12:24:49.706956 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:49.706958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:49.706996 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:49.707061 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:49.707514 (+ 453us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58344: BlockingRecv error: recv got EOF from 127.0.0.1:58344 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260619 12:24:49.707708 7071 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1781871857195028-7071-0/krb5kdc/krb5cc' of type FILE
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:49.708859 7071 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:49.710053 7071 init.cc:295] Successfully reacquired a new kerberos TGT
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[12603](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871889, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
I20260619 12:24:49.719924 13114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:49.711015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58346 (local address 127.6.231.193:36405)
0619 12:24:49.711173 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:49.711179 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:49.711255 (+ 76us) server_negotiation.cc:407] Connection header received
0619 12:24:49.711445 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:49.711449 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:49.711500 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:49.711606 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:49.712219 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.712784 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.713771 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:49.714139 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:49.714301 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:49.716716 (+ 2415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:49.716718 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:49.716720 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:49.716745 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:49.718395 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:49.718842 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:49.718845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:49.718846 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:49.718923 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:49.719195 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:49.719197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:49.719199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:49.719364 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:49.719462 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:49.719700 (+ 238us) server_negotiation.cc:299] Negotiation successful
0619 12:24:49.719796 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 12:24:49.721067 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 12710
I20260619 12:24:49.726773 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 12844
I20260619 12:24:49.732254 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 12978
I20260619 12:24:49.737612 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 12616
2026-06-19T12:24:49Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2210 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[13127](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[13127](info): set up 2 sockets
Jun 19 12:24:49 dist-test-slave-kr9h krb5kdc[13127](info): commencing operation
krb5kdc: starting...
W20260619 12:24:50.781898 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.022s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:24:50 dist-test-slave-kr9h krb5kdc[13127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:24:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:50Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.254.keytab.
Entry for principal oryx/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.254.keytab.
I20260619 12:24:50.904350 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:45495
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.254.keytab
--principal=oryx/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:45495
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:51.014298 13140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:51.014593 13140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:51.014634 13140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:51.018594 13140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:51.018653 13140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:51.018672 13140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:51.018689 13140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:51.018704 13140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:51.023934 13140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:45495
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.254.keytab
--principal=oryx/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:45495
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:51.024957 13140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:51.025950 13140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:51.034403 13145 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:51.034446 13148 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:51.034407 13146 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:51.034840 13140 server_base.cc:1034] running on GCE node
I20260619 12:24:51.035380 13140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:51.036425 13140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:51.037613 13140 hybrid_clock.cc:648] HybridClock initialized: now 1781871891037584 us; error 49 us; skew 500 ppm
Jun 19 12:24:51 dist-test-slave-kr9h krb5kdc[13127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871891, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:51.049160 13140 init.cc:377] Logged in from keytab as oryx/127.6.231.254@KRBTEST.COM (short username oryx)
I20260619 12:24:51.050271 13140 webserver.cc:460] Webserver started at http://127.6.231.254:35989/ using document root <none> and password file <none>
I20260619 12:24:51.050871 13140 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:51.050964 13140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:51.051208 13140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:51.053305 13140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "b068dd17324e4c2a8cf783d248709eef"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "16e5bf1ac23ce91cb55a73d7a53a06b2"
server_key_iv: "70b4a38f51e0bf0aec016f804fb35821"
server_key_version: "encryptionkey@0"
I20260619 12:24:51.053936 13140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "b068dd17324e4c2a8cf783d248709eef"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "16e5bf1ac23ce91cb55a73d7a53a06b2"
server_key_iv: "70b4a38f51e0bf0aec016f804fb35821"
server_key_version: "encryptionkey@0"
I20260619 12:24:51.057685 13140 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 12:24:51.060180 13155 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:51.061581 13140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:51.061688 13140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "b068dd17324e4c2a8cf783d248709eef"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "16e5bf1ac23ce91cb55a73d7a53a06b2"
server_key_iv: "70b4a38f51e0bf0aec016f804fb35821"
server_key_version: "encryptionkey@0"
I20260619 12:24:51.061775 13140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:51.087005 13140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:51.089977 13140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:51.090128 13140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:51.096798 13140 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:45495
I20260619 12:24:51.097081 13207 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:45495 every 8 connection(s)
I20260619 12:24:51.097801 13140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:51.100711 13208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:51.101184 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 13140
I20260619 12:24:51.101336 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:51.101678 7071 external_mini_cluster.cc:1428] Setting key 3ccf9530e816c3369f7059fd8f102c98
I20260619 12:24:51.107749 13208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef: Bootstrap starting.
Jun 19 12:24:51 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
I20260619 12:24:51.110464 13208 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:51.111302 13208 log.cc:824] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:51.113256 13208 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef: No bootstrap required, opened a new log
I20260619 12:24:51.116879 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:51.103041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59716 (local address 127.6.231.254:45495)
0619 12:24:51.103435 (+ 394us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:51.103451 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:51.103485 (+ 34us) server_negotiation.cc:407] Connection header received
0619 12:24:51.104141 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:51.104157 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:51.104475 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:51.104742 (+ 267us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:51.105637 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.106607 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.107715 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.108221 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.108345 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:51.110991 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:51.110997 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:51.111012 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:51.111040 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:51.114148 (+ 3108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:51.114598 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:51.114604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:51.114608 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:51.114714 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:51.115003 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:51.115006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:51.115008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:51.115396 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:51.115596 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:51.115882 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 12:24:51.116154 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":127,"threads_started":1}
I20260619 12:24:51.116850 13208 raft_consensus.cc:348] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } }
I20260619 12:24:51.117156 13208 raft_consensus.cc:374] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:51.117231 13208 raft_consensus.cc:729] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b068dd17324e4c2a8cf783d248709eef, State: Initialized, Role: FOLLOWER
I20260619 12:24:51.117710 13208 consensus_queue.cc:260] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } }
I20260619 12:24:51.117880 13208 raft_consensus.cc:388] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:51.117964 13208 raft_consensus.cc:482] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:51.118065 13208 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:51.119015 13208 raft_consensus.cc:504] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } }
I20260619 12:24:51.119345 13208 leader_election.cc:302] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b068dd17324e4c2a8cf783d248709eef; no voters:
I20260619 12:24:51.119644 13208 leader_election.cc:288] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:51.119791 13213 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:51.120039 13213 raft_consensus.cc:686] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [term 1 LEADER]: Becoming Leader. State: Replica: b068dd17324e4c2a8cf783d248709eef, State: Running, Role: LEADER
I20260619 12:24:51.120371 13213 consensus_queue.cc:237] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } }
I20260619 12:24:51.120877 13208 sys_catalog.cc:564] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:51.122164 13215 sys_catalog.cc:455] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: SysCatalogTable state changed. Reason: New leader b068dd17324e4c2a8cf783d248709eef. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } }
I20260619 12:24:51.122205 13214 sys_catalog.cc:455] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } }
I20260619 12:24:51.122298 13215 sys_catalog.cc:458] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:51.122298 13214 sys_catalog.cc:458] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:51.122714 13222 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:51.125847 13222 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:51.131403 13222 catalog_manager.cc:1347] Generated new cluster ID: f195b60636ac4e59acfdbbf28c05f478
I20260619 12:24:51.131500 13222 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:51.137967 13222 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:51.138971 13222 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:51.144922 13222 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef: Generated new TSK 0
I20260619 12:24:51.145638 13222 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab.
Entry for principal oryx/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab.
I20260619 12:24:51.176342 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45495
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab
--principal=oryx/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:51.292320 13234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:51.292582 13234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:51.292625 13234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:51.296471 13234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:51.296523 13234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:51.296579 13234 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:24:51.301736 13234 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab
--principal=oryx/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:51.302776 13234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:51.303874 13234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:51.313311 13242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:51.313411 13240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:51.313319 13239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:51.313621 13234 server_base.cc:1034] running on GCE node
I20260619 12:24:51.314134 13234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:51.314781 13234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:51.315955 13234 hybrid_clock.cc:648] HybridClock initialized: now 1781871891315939 us; error 27 us; skew 500 ppm
Jun 19 12:24:51 dist-test-slave-kr9h krb5kdc[13127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871891, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:51.318789 13234 init.cc:377] Logged in from keytab as oryx/127.6.231.193@KRBTEST.COM (short username oryx)
I20260619 12:24:51.319698 13234 webserver.cc:460] Webserver started at http://127.6.231.193:33775/ using document root <none> and password file <none>
I20260619 12:24:51.320262 13234 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:51.320393 13234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:51.320641 13234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:51.322988 13234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "be0bb06130374b22bbf99f9eb10545ce"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "de9ced78daa5b69f61da7450e9497f47"
server_key_iv: "4c0c794849da42bed936994b39386992"
server_key_version: "encryptionkey@0"
I20260619 12:24:51.323542 13234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "be0bb06130374b22bbf99f9eb10545ce"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "de9ced78daa5b69f61da7450e9497f47"
server_key_iv: "4c0c794849da42bed936994b39386992"
server_key_version: "encryptionkey@0"
I20260619 12:24:51.327318 13234 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:24:51.329885 13249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:51.331250 13234 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:24:51.331398 13234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "be0bb06130374b22bbf99f9eb10545ce"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "de9ced78daa5b69f61da7450e9497f47"
server_key_iv: "4c0c794849da42bed936994b39386992"
server_key_version: "encryptionkey@0"
I20260619 12:24:51.331522 13234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:51.345664 13234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:51.349179 13234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:51.349416 13234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:51.350133 13234 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:51.350965 13234 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:51.351038 13234 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:51.351110 13234 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:51.351159 13234 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:51.360234 13234 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:38201
I20260619 12:24:51.360252 13362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:38201 every 8 connection(s)
I20260619 12:24:51.361168 13234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:24:51.362982 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 13234
I20260619 12:24:51.363101 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:24:51.363426 7071 external_mini_cluster.cc:1428] Setting key f4b6c752f08f9cb54bf05e7ac363556d
Jun 19 12:24:51 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871891, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.6.231.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.231.194@KRBTEST.COM" created.
I20260619 12:24:51.375723 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:51.363099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:44659 (local address 127.6.231.254:45495)
0619 12:24:51.363247 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:51.363254 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:51.364216 (+ 962us) server_negotiation.cc:407] Connection header received
0619 12:24:51.365667 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:51.365673 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:51.365736 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:51.365827 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:51.366994 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.367528 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.368674 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.368924 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.369036 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:51.371827 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:51.371831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:51.371834 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:51.371871 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:51.373596 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:51.374110 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:51.374112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:51.374114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:51.374182 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:51.374499 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:51.374501 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:51.374503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:51.374739 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:51.374882 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:51.375432 (+ 550us) server_negotiation.cc:299] Negotiation successful
0619 12:24:51.375559 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 12:24:51.376592 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:51.363558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.193:44659)
0619 12:24:51.364061 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:51.364115 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:51.365389 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:51.365994 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:51.366001 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:51.366355 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:51.366839 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:51.366853 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.367656 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.367660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:51.368548 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:51.368557 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.369072 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.369077 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:51.369253 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:51.369972 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:51.369988 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:51.371635 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:51.373701 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:51.373706 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:51.373718 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:51.373985 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:51.374257 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:51.374260 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:51.374262 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:51.374411 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:51.374878 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:51.374887 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:51.375140 (+ 253us) client_negotiation.cc:769] Sending connection context
0619 12:24:51.375393 (+ 253us) client_negotiation.cc:240] Negotiation successful
0619 12:24:51.375728 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":135,"threads_started":1}
I20260619 12:24:51.378016 13363 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45495
I20260619 12:24:51.378275 13363 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:51.378904 13363 heartbeater.cc:511] Master 127.6.231.254:45495 requested a full tablet report, sending...
I20260619 12:24:51.380728 13172 ts_manager.cc:194] Registered new tserver with Master: be0bb06130374b22bbf99f9eb10545ce (127.6.231.193:38201)
I20260619 12:24:51.382414 13172 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:44659
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab.
Entry for principal oryx/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab.
I20260619 12:24:51.390421 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45495
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab
--principal=oryx/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:51.506956 13368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:51.507231 13368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:51.507287 13368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:51.511601 13368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:51.511663 13368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:51.511722 13368 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:24:51.517030 13368 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab
--principal=oryx/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:51.518177 13368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:51.519176 13368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:51.529183 13374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:51.529297 13368 server_base.cc:1034] running on GCE node
W20260619 12:24:51.529183 13376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:51.529193 13373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:51.529942 13368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:51.530623 13368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:51.531781 13368 hybrid_clock.cc:648] HybridClock initialized: now 1781871891531750 us; error 44 us; skew 500 ppm
Jun 19 12:24:51 dist-test-slave-kr9h krb5kdc[13127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871891, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:51.534557 13368 init.cc:377] Logged in from keytab as oryx/127.6.231.194@KRBTEST.COM (short username oryx)
I20260619 12:24:51.535476 13368 webserver.cc:460] Webserver started at http://127.6.231.194:42567/ using document root <none> and password file <none>
I20260619 12:24:51.536072 13368 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:51.536180 13368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:51.536404 13368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:51.538478 13368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "e3d09f2dbcb2c77f094699ad7a978f9a"
server_key_iv: "d00463240f61316782d51a104052a083"
server_key_version: "encryptionkey@0"
I20260619 12:24:51.539052 13368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "e3d09f2dbcb2c77f094699ad7a978f9a"
server_key_iv: "d00463240f61316782d51a104052a083"
server_key_version: "encryptionkey@0"
I20260619 12:24:51.542701 13368 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:24:51.545120 13383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:51.546548 13368 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:24:51.546782 13368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "e3d09f2dbcb2c77f094699ad7a978f9a"
server_key_iv: "d00463240f61316782d51a104052a083"
server_key_version: "encryptionkey@0"
I20260619 12:24:51.546900 13368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:51.572012 13368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:51.575357 13368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:51.575560 13368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:51.576156 13368 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:51.576970 13368 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:51.577039 13368 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:51.577113 13368 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:51.577167 13368 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:51.585974 13368 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:43219
I20260619 12:24:51.586033 13496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:43219 every 8 connection(s)
I20260619 12:24:51.586896 13368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:24:51.587096 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 13368
I20260619 12:24:51.587214 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:24:51.587602 7071 external_mini_cluster.cc:1428] Setting key c9fab5079698ed55236cb38750bda5b0
Jun 19 12:24:51 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871891, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for oryx/127.6.231.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.231.195@KRBTEST.COM" created.
I20260619 12:24:51.601001 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:51.588723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:57851 (local address 127.6.231.254:45495)
0619 12:24:51.588881 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:51.588885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:51.590081 (+ 1196us) server_negotiation.cc:407] Connection header received
0619 12:24:51.590959 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:51.590963 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:51.591013 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:51.591094 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:51.592245 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.592790 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.594005 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.594250 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.594369 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:51.597028 (+ 2659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:51.597032 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:51.597035 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:51.597064 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:51.598579 (+ 1515us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:51.599195 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:51.599202 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:51.599206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:51.599311 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:51.599731 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:51.599734 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:51.599735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:51.599914 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:51.599992 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:51.600641 (+ 649us) server_negotiation.cc:299] Negotiation successful
0619 12:24:51.600783 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260619 12:24:51.601874 13499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:51.589234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.194:57851)
0619 12:24:51.589939 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:51.589983 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:51.590754 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:51.591272 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:51.591279 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:51.591625 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:51.592093 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:51.592106 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.592917 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.592923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:51.593881 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:51.593891 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.594403 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.594407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:51.594571 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:51.595185 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:51.595200 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:51.596861 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:51.598682 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:51.598686 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:51.598698 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:51.599036 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:51.599442 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:51.599445 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:51.599447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:51.599632 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:51.600019 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:51.600029 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:51.600354 (+ 325us) client_negotiation.cc:769] Sending connection context
0619 12:24:51.600572 (+ 218us) client_negotiation.cc:240] Negotiation successful
0619 12:24:51.600922 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":149,"threads_started":1}
I20260619 12:24:51.603013 13497 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45495
I20260619 12:24:51.603281 13497 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:51.603760 13497 heartbeater.cc:511] Master 127.6.231.254:45495 requested a full tablet report, sending...
I20260619 12:24:51.604791 13172 ts_manager.cc:194] Registered new tserver with Master: 26bc0b6a6c4a4e089eff44ddf15bcc05 (127.6.231.194:43219)
I20260619 12:24:51.605391 13172 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:57851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab.
Entry for principal oryx/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab.
I20260619 12:24:51.614920 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45495
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab
--principal=oryx/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:51.724635 13502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:51.724923 13502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:51.724965 13502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:51.729125 13502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:51.729194 13502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:51.729337 13502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:24:51.735278 13502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab
--principal=oryx/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45495
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:51.736434 13502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:51.737541 13502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:51.747125 13510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:51.747126 13507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:51.747141 13508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:51.747538 13502 server_base.cc:1034] running on GCE node
I20260619 12:24:51.747963 13502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:51.748589 13502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:51.749751 13502 hybrid_clock.cc:648] HybridClock initialized: now 1781871891749735 us; error 29 us; skew 500 ppm
Jun 19 12:24:51 dist-test-slave-kr9h krb5kdc[13127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871891, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:51.752439 13502 init.cc:377] Logged in from keytab as oryx/127.6.231.195@KRBTEST.COM (short username oryx)
I20260619 12:24:51.753273 13502 webserver.cc:460] Webserver started at http://127.6.231.195:41725/ using document root <none> and password file <none>
I20260619 12:24:51.753844 13502 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:51.753904 13502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:51.754062 13502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:51.756029 13502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "75adfcbbfdae4fbc91f0e4089736c0dd"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "ff5192e777c0bc0f4b1f9554dbf7ef9b"
server_key_iv: "7df7f569a71515116dbc5597da95eb43"
server_key_version: "encryptionkey@0"
I20260619 12:24:51.756521 13502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "75adfcbbfdae4fbc91f0e4089736c0dd"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "ff5192e777c0bc0f4b1f9554dbf7ef9b"
server_key_iv: "7df7f569a71515116dbc5597da95eb43"
server_key_version: "encryptionkey@0"
I20260619 12:24:51.760113 13502 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260619 12:24:51.762598 13517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:51.763948 13502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:51.764075 13502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "75adfcbbfdae4fbc91f0e4089736c0dd"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "ff5192e777c0bc0f4b1f9554dbf7ef9b"
server_key_iv: "7df7f569a71515116dbc5597da95eb43"
server_key_version: "encryptionkey@0"
I20260619 12:24:51.764179 13502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:51.803769 13502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:51.807061 13502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:51.807277 13502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:51.807894 13502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:51.808719 13502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:51.808787 13502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:51.808853 13502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:51.808898 13502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:51.817977 13502 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:34249
I20260619 12:24:51.817998 13630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:34249 every 8 connection(s)
I20260619 12:24:51.818862 13502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:24:51.821736 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 13502
I20260619 12:24:51.821866 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:24:51.822250 7071 external_mini_cluster.cc:1428] Setting key d57bb8cd5dea96256135bf7ef1ddc5b1
Jun 19 12:24:51 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871891, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
I20260619 12:24:51.832567 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:51.820684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:44891 (local address 127.6.231.254:45495)
0619 12:24:51.820838 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:51.820843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:51.822183 (+ 1340us) server_negotiation.cc:407] Connection header received
0619 12:24:51.823179 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:51.823182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:51.823239 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:51.823320 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:51.824478 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.825034 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.826271 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.826506 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.826620 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:51.829025 (+ 2405us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:51.829028 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:51.829032 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:51.829060 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:51.830550 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:51.831079 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:51.831082 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:51.831083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:51.831162 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:51.831525 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:51.831527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:51.831529 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:51.831786 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:51.831873 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:51.832359 (+ 486us) server_negotiation.cc:299] Negotiation successful
0619 12:24:51.832450 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:24:51.833567 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:51.821160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.195:44891)
0619 12:24:51.821987 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:51.822064 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:51.822973 (+ 909us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:51.823495 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:51.823502 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:51.823862 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:51.824341 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:51.824353 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.825164 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.825167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:51.826146 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:51.826155 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:51.826656 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:51.826660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:51.826827 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:51.827412 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:51.827427 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:51.828868 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:51.830683 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:51.830688 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:51.830699 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:51.830967 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:51.831262 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:51.831265 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:51.831266 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:51.831407 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:51.831904 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:51.831909 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:51.832135 (+ 226us) client_negotiation.cc:769] Sending connection context
0619 12:24:51.832311 (+ 176us) client_negotiation.cc:240] Negotiation successful
0619 12:24:51.832553 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":612,"thread_start_us":210,"threads_started":1}
I20260619 12:24:51.834836 13631 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45495
I20260619 12:24:51.835103 13631 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:51.835654 13631 heartbeater.cc:511] Master 127.6.231.254:45495 requested a full tablet report, sending...
I20260619 12:24:51.836753 13172 ts_manager.cc:194] Registered new tserver with Master: 75adfcbbfdae4fbc91f0e4089736c0dd (127.6.231.195:34249)
I20260619 12:24:51.837371 13172 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:44891
I20260619 12:24:51.848415 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.231.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.231.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.231.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.253.keytab.
Entry for principal oryx/127.6.231.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.253.keytab.
I20260619 12:24:51.877080 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.253:38667
--webserver_interface=127.6.231.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.253.keytab
--principal=oryx/127.6.231.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:45495,127.6.231.253:38667
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:51.991645 13636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:51.991969 13636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:51.992053 13636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:51.996042 13636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:51.996135 13636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:51.996174 13636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:51.996212 13636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:51.996250 13636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:52.001653 13636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:45495,127.6.231.253:38667
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.253.keytab
--principal=oryx/127.6.231.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.253:38667
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.231.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
W20260619 12:24:52.001917 13636 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.6.231.254:45495,127.6.231.253:38667'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260619 12:24:52.002813 13636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:52.003818 13636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:52.012372 13644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:52.012380 13641 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:52.012722 13636 server_base.cc:1034] running on GCE node
W20260619 12:24:52.012380 13642 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:52.013420 13636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:52.014863 13636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:52.016063 13636 hybrid_clock.cc:648] HybridClock initialized: now 1781871892016050 us; error 29 us; skew 500 ppm
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:52.019136 13636 init.cc:377] Logged in from keytab as oryx/127.6.231.253@KRBTEST.COM (short username oryx)
I20260619 12:24:52.020242 13636 webserver.cc:460] Webserver started at http://127.6.231.253:37163/ using document root <none> and password file <none>
I20260619 12:24:52.020821 13636 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:52.020941 13636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:52.021179 13636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:52.023514 13636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/data/instance:
uuid: "7e962ed1f9c84322b912231dd6c821f5"
format_stamp: "Formatted at 2026-06-19 12:24:52 on dist-test-slave-kr9h"
server_key: "682ddf12143ebdc326f121504bdc843c"
server_key_iv: "703f53e545ab11902b761d21e221ac6c"
server_key_version: "encryptionkey@0"
I20260619 12:24:52.024004 13636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/wal/instance:
uuid: "7e962ed1f9c84322b912231dd6c821f5"
format_stamp: "Formatted at 2026-06-19 12:24:52 on dist-test-slave-kr9h"
server_key: "682ddf12143ebdc326f121504bdc843c"
server_key_iv: "703f53e545ab11902b761d21e221ac6c"
server_key_version: "encryptionkey@0"
I20260619 12:24:52.027733 13636 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:24:52.030117 13651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:52.031414 13636 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:24:52.031560 13636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/wal
uuid: "7e962ed1f9c84322b912231dd6c821f5"
format_stamp: "Formatted at 2026-06-19 12:24:52 on dist-test-slave-kr9h"
server_key: "682ddf12143ebdc326f121504bdc843c"
server_key_iv: "703f53e545ab11902b761d21e221ac6c"
server_key_version: "encryptionkey@0"
I20260619 12:24:52.031677 13636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:52.049415 13636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:52.052783 13636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:52.052937 13636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:52.059796 13636 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.253:38667
I20260619 12:24:52.060098 13703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.253:38667 every 8 connection(s)
I20260619 12:24:52.060846 13636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/data/info.pb
I20260619 12:24:52.063562 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 13636
I20260619 12:24:52.063800 13704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:52.068697 13704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.231.253@KRBTEST.COM
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.253@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
I20260619 12:24:52.079632 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.065123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51954 (local address 127.6.231.253:38667)
0619 12:24:52.065671 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.065688 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.065723 (+ 35us) server_negotiation.cc:407] Connection header received
0619 12:24:52.066646 (+ 923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.066663 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.067009 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.067457 (+ 448us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.068384 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.069556 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.070434 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.071096 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.071272 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.074121 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.074127 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.074136 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.074161 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.076162 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.076623 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.076628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.076632 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.076726 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.077029 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.077034 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.077036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.077603 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.077806 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.078145 (+ 339us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.078479 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":154,"threads_started":1}
I20260619 12:24:52.080258 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.070614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59718 (local address 127.6.231.254:45495)
0619 12:24:52.070756 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.070761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.071040 (+ 279us) server_negotiation.cc:407] Connection header received
0619 12:24:52.071165 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.071171 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.071225 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.071302 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.072145 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.072652 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.073754 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.073989 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.074103 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.076621 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.076627 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.076635 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.076658 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.078449 (+ 1791us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.079011 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.079014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.079016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.079090 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.079397 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.079400 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.079401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.079595 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.079672 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.080054 (+ 382us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.080151 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 12:24:52.080323 13708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.070500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:59718)
0619 12:24:52.070903 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.070950 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.071068 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.071370 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.071375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.071836 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.072012 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.072021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.072794 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.072800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.073624 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.073637 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.074121 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.074125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.074248 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.074965 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.074976 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.076485 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.078554 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.078561 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.078567 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.078864 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.079202 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.079205 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.079207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.079304 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.079705 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.079712 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.079836 (+ 124us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.079984 (+ 148us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.080152 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":124,"threads_started":1}
I20260619 12:24:52.081624 13704 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.253@KRBTEST.COM for oryx/127.6.231.253@KRBTEST.COM
I20260619 12:24:52.090571 13708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.082118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:38667 (local address 127.0.0.1:51968)
0619 12:24:52.082309 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.082323 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.082408 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.082602 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.082604 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.082777 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.082855 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.082860 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.083896 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.083899 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.084401 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.084407 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.084839 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.084842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.084967 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.085394 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.085405 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.086713 (+ 1308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.088668 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.088703 (+ 35us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.088707 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.089018 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.089533 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.089536 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.089538 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.089619 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.090008 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.090011 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.090089 (+ 78us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.090197 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.090337 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65,"mutex_wait_us":21}
I20260619 12:24:52.090571 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.082207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51968 (local address 127.6.231.253:38667)
0619 12:24:52.082333 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.082338 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.082371 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:24:52.082474 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.082479 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.082512 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.082596 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.082956 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.083783 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.084504 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.084740 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.084842 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.086888 (+ 2046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.086891 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.086893 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.086915 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.088563 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.089315 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.089319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.089321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.089422 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.089707 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.089713 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.089715 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.089910 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.090010 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.090237 (+ 227us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.090385 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":71,"server-negotiator.queue_time_us":73}
I20260619 12:24:52.093706 13704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: Bootstrap starting.
I20260619 12:24:52.095870 13704 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:52.096539 13704 log.cc:824] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:52.098501 13704 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: No bootstrap required, opened a new log
I20260619 12:24:52.101516 13704 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } }
I20260619 12:24:52.101696 13704 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:52.101765 13704 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e962ed1f9c84322b912231dd6c821f5, State: Initialized, Role: FOLLOWER
I20260619 12:24:52.102175 13704 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } }
I20260619 12:24:52.102612 13710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } } }
I20260619 12:24:52.102738 13710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:24:52.103358 13704 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: configured and running, proceeding with master startup.
W20260619 12:24:52.107893 13721 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:24:52.108033 13721 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:24:52.109722 13636 master_runner.cc:428] Detected that this master 7e962ed1f9c84322b912231dd6c821f5 is joining an existing cluster
I20260619 12:24:52.109777 13636 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.231.253:38667
I20260619 12:24:52.120886 13727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.112159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:59722)
0619 12:24:52.112629 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.112644 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.112799 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.113156 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.113159 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.113525 (+ 366us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.113703 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.113723 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.114756 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.114759 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.115423 (+ 664us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.115430 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.116039 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.116042 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.116204 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.116804 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.116818 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.117175 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.119195 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.119200 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.119202 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.119443 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.119734 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.119740 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.119743 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.119849 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.120328 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.120333 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.120433 (+ 100us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.120539 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.120672 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":130,"thread_start_us":75,"threads_started":1}
I20260619 12:24:52.120958 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.112232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.6.231.254:45495)
0619 12:24:52.112374 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.112379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.112715 (+ 336us) server_negotiation.cc:407] Connection header received
0619 12:24:52.112912 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.112915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.112964 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.113015 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.113872 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.114634 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.115545 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.115788 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.115906 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.117310 (+ 1404us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.117314 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.117316 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.117342 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.119062 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.119541 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.119547 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.119549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.119635 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.119957 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.119960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.119962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.120185 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.120282 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.120613 (+ 331us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.120785 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:24:52.121336 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.112573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51984 (local address 127.6.231.253:38667)
0619 12:24:52.112688 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.112695 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.112905 (+ 210us) server_negotiation.cc:407] Connection header received
0619 12:24:52.113036 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.113041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.113079 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.113174 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.113806 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.114576 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.115865 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.116255 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.116356 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.117593 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.117597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.117599 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.117619 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.119433 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.120117 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.120123 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.120125 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.120198 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.120459 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.120462 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.120464 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.120662 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.120751 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.121013 (+ 262us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.121114 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":66,"server-negotiator.queue_time_us":41}
I20260619 12:24:52.121748 13728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.112515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:38667 (local address 127.0.0.1:51984)
0619 12:24:52.112814 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.112825 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.112936 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.113189 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.113191 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.113561 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.113698 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.113705 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.114704 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.114708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.115735 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.115756 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.116371 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.116374 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.116500 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.116984 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.116999 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.117463 (+ 464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.119770 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.119806 (+ 36us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.119808 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.120031 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.120285 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.120288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.120290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.120367 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.120783 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.120785 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.120861 (+ 76us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.121369 (+ 508us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.121528 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":47,"thread_start_us":72,"threads_started":1}
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871891, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.254@KRBTEST.COM for oryx/127.6.231.253@KRBTEST.COM
I20260619 12:24:52.136565 13707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.126202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51996 (local address 127.6.231.253:38667)
0619 12:24:52.126813 (+ 611us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.126817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.127182 (+ 365us) server_negotiation.cc:407] Connection header received
0619 12:24:52.127370 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.127373 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.127413 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.127479 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.128236 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.128706 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.129994 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.130192 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.130276 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.132758 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.132761 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.132763 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.132783 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.134427 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.135042 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.135049 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.135051 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.135162 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.135463 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.135468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.135470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.135699 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.135808 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.136317 (+ 509us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.136457 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529}
I20260619 12:24:52.137024 13729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.126158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:38667 (local address 127.0.0.1:51996)
0619 12:24:52.127047 (+ 889us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.127101 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.127241 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.127536 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.127543 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.128000 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.128127 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.128135 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.128808 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.128812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.129870 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.129880 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.130305 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.130308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.130416 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.130944 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.130959 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.132613 (+ 1654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.134543 (+ 1930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.134555 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.134566 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.134940 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.135270 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.135273 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.135278 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.135370 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.135915 (+ 545us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.135921 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.136024 (+ 103us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.136722 (+ 698us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.136904 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":90,"threads_started":1}
I20260619 12:24:52.138391 13172 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.231.253:38667: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5"
member_type: NON_VOTER
last_known_addr {
host: "127.6.231.253"
port: 38667
}
attrs {
promote: true
}
}
dest_uuid: "b068dd17324e4c2a8cf783d248709eef"
cas_config_opid_index: -1
I20260619 12:24:52.139051 13172 consensus_queue.cc:237] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true } }
I20260619 12:24:52.140650 13731 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 7e962ed1f9c84322b912231dd6c821f5 (127.6.231.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true } } }
I20260619 12:24:52.141626 13678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:52.142445 13731 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.231.253:38667
I20260619 12:24:52.142841 13732 sys_catalog.cc:455] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true } } }
I20260619 12:24:52.142966 13678 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 FOLLOWER]: Refusing update from remote peer b068dd17324e4c2a8cf783d248709eef: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260619 12:24:52.143064 13636 master.cc:537] Master@127.6.231.253:38667 shutting down...
WARNING: no policy specified for oryx/127.6.231.252@KRBTEST.COM; defaulting to no policy
I20260619 12:24:52.145557 13731 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:52.145692 13732 sys_catalog.cc:458] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: This master's current role is: LEADER
W20260619 12:24:52.147670 13159 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260619 12:24:52.147719 13636 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:52.148002 13636 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 FOLLOWER]: Raft consensus is shut down!
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.231.252@KRBTEST.COM" created.
I20260619 12:24:52.148154 13636 tablet_replica.cc:330] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: stopping tablet replica
W20260619 12:24:52.148869 13159 consensus_peers.cc:487] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef -> Peer 7e962ed1f9c84322b912231dd6c821f5 (127.6.231.253:38667): Couldn't send request to peer 7e962ed1f9c84322b912231dd6c821f5. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 12:24:52.152771 13636 master.cc:552] Master@127.6.231.253:38667 shutdown complete.
I20260619 12:24:52.152896 13636 master_runner.cc:305] Clearing existing system tablet
I20260619 12:24:52.153205 13636 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:52.153292 13636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:52.154922 13636 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:52.155877 13737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:52.156091 13636 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:52.156204 13636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:52.156236 13636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/wal
uuid: "7e962ed1f9c84322b912231dd6c821f5"
format_stamp: "Formatted at 2026-06-19 12:24:52 on dist-test-slave-kr9h"
server_key: "682ddf12143ebdc326f121504bdc843c"
server_key_iv: "703f53e545ab11902b761d21e221ac6c"
server_key_version: "encryptionkey@0"
I20260619 12:24:52.158382 13636 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:52.161549 13636 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 12:24:52.161641 13636 log.cc:1196] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260619 12:24:52.161892 13636 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: Deleting consensus metadata
I20260619 12:24:52.162060 13636 master_runner.cc:315] Copying system tablet from 127.6.231.254:45495
I20260619 12:24:52.163209 13636 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: tablet copy: Beginning tablet copy session from remote peer at address 127.6.231.254:45495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.231.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.252.keytab.
Entry for principal oryx/127.6.231.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.252.keytab.
I20260619 12:24:52.166728 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.252:33277
--webserver_interface=127.6.231.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.252.keytab
--principal=oryx/127.6.231.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:45495,127.6.231.253:38667,127.6.231.252:33277
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:24:52.172811 13743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.163830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:59726)
0619 12:24:52.164208 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.164222 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.164377 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.164651 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.164654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.164886 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.164996 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.165002 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.166103 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.166107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.167063 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.167078 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.167759 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.167763 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.167855 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.168431 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.168446 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.168955 (+ 509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.171014 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.171017 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.171019 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.171260 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.171589 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.171591 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.171593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.171686 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.172179 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.172184 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.172301 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.172446 (+ 145us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.172591 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":104,"threads_started":1}
I20260619 12:24:52.173032 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.163915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59726 (local address 127.6.231.254:45495)
0619 12:24:52.164064 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.164069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.164325 (+ 256us) server_negotiation.cc:407] Connection header received
0619 12:24:52.164493 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.164496 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.164542 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.164635 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.165120 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.165978 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.167216 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.167626 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.167758 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.169113 (+ 1355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.169117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.169120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.169146 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.170889 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.171368 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.171374 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.171378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.171481 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.171797 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.171804 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.171808 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.172072 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.172188 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.172522 (+ 334us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.172673 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 12:24:52.173563 13192 tablet_copy_service.cc:140] P b068dd17324e4c2a8cf783d248709eef: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 7e962ed1f9c84322b912231dd6c821f5 ({username='oryx', principal='oryx/127.6.231.253@KRBTEST.COM'} at 127.0.0.1:59726)
I20260619 12:24:52.173758 13192 tablet_copy_service.cc:161] P b068dd17324e4c2a8cf783d248709eef: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 7e962ed1f9c84322b912231dd6c821f5 at {username='oryx', principal='oryx/127.6.231.253@KRBTEST.COM'} at 127.0.0.1:59726: session id = 7e962ed1f9c84322b912231dd6c821f5-00000000000000000000000000000000
I20260619 12:24:52.175546 13192 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 12:24:52.177500 13636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 12:24:52.180166 13636 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: tablet copy: Starting download of 0 data blocks...
I20260619 12:24:52.180364 13636 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: tablet copy: Starting download of 1 WAL segments...
I20260619 12:24:52.182271 13636 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 12:24:52.184197 13192 tablet_copy_service.cc:342] P b068dd17324e4c2a8cf783d248709eef: Request end of tablet copy session 7e962ed1f9c84322b912231dd6c821f5-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.231.253@KRBTEST.COM'} at 127.0.0.1:59726
I20260619 12:24:52.184341 13192 tablet_copy_service.cc:433] P b068dd17324e4c2a8cf783d248709eef: ending tablet copy session 7e962ed1f9c84322b912231dd6c821f5-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 7e962ed1f9c84322b912231dd6c821f5
I20260619 12:24:52.192641 13636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:52.194321 13745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:52.194438 13748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:52.194569 13746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:52.197966 13636 server_base.cc:1034] running on GCE node
I20260619 12:24:52.198130 13636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:52.198432 13636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:52.199575 13636 hybrid_clock.cc:648] HybridClock initialized: now 1781871892199552 us; error 41 us; skew 500 ppm
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:52.201248 13636 init.cc:377] Logged in from keytab as oryx/127.6.231.253@KRBTEST.COM (short username oryx)
I20260619 12:24:52.201735 13636 webserver.cc:460] Webserver started at http://127.6.231.253:41939/ using document root <none> and password file <none>
I20260619 12:24:52.201936 13636 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:52.201979 13636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:52.203271 13636 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260619 12:24:52.204245 13755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:52.204486 13636 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:52.204592 13636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/wal
uuid: "7e962ed1f9c84322b912231dd6c821f5"
format_stamp: "Formatted at 2026-06-19 12:24:52 on dist-test-slave-kr9h"
server_key: "682ddf12143ebdc326f121504bdc843c"
server_key_iv: "703f53e545ab11902b761d21e221ac6c"
server_key_version: "encryptionkey@0"
I20260619 12:24:52.204746 13636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:52.212317 13636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:52.214516 13636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:52.219267 13636 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.253:38667
I20260619 12:24:52.219309 13807 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.253:38667 every 8 connection(s)
I20260619 12:24:52.219925 13636 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-1/data/info.pb
I20260619 12:24:52.221820 13808 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:24:52.222826 13808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: Bootstrap starting.
I20260619 12:24:52.232800 13808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:24:52.233103 13808 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: Bootstrap complete.
I20260619 12:24:52.233723 13808 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true } }
I20260619 12:24:52.233863 13808 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 7e962ed1f9c84322b912231dd6c821f5, State: Initialized, Role: LEARNER
I20260619 12:24:52.233966 13808 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true } }
I20260619 12:24:52.234301 13811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true } } }
I20260619 12:24:52.234483 13811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: This master's current role is: LEARNER
I20260619 12:24:52.234512 13808 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:52.237839 13822 catalog_manager.cc:1259] Loaded cluster ID: f195b60636ac4e59acfdbbf28c05f478
I20260619 12:24:52.237960 13822 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: loading cluster ID for follower catalog manager: success
I20260619 12:24:52.240051 13822 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: acquiring CA information for follower catalog manager: success
I20260619 12:24:52.241420 13822 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260619 12:24:52.297991 13744 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:52.298291 13744 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:52.298373 13744 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:52.302541 13744 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:52.302619 13744 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:52.302656 13744 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:52.302709 13744 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:52.302757 13744 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:52.308511 13744 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:45495,127.6.231.253:38667,127.6.231.252:33277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.252.keytab
--principal=oryx/127.6.231.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.252:33277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.231.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13744
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:52.309736 13744 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:52.310830 13744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:52.319344 13830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:52.319442 13828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:52.319458 13827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:52.319557 13744 server_base.cc:1034] running on GCE node
I20260619 12:24:52.320194 13744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:52.321352 13744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:52.322518 13744 hybrid_clock.cc:648] HybridClock initialized: now 1781871892322474 us; error 59 us; skew 500 ppm
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:52.325354 13744 init.cc:377] Logged in from keytab as oryx/127.6.231.252@KRBTEST.COM (short username oryx)
I20260619 12:24:52.326411 13744 webserver.cc:460] Webserver started at http://127.6.231.252:41727/ using document root <none> and password file <none>
I20260619 12:24:52.326941 13744 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:52.327000 13744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:52.327160 13744 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:52.329128 13744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/data/instance:
uuid: "86026bdd073e435980fb26cf963f9f1b"
format_stamp: "Formatted at 2026-06-19 12:24:52 on dist-test-slave-kr9h"
server_key: "ef12de90260669570b8fbd9faea1f8aa"
server_key_iv: "c3cd63294fdbf44a79a6f37dd8101d5d"
server_key_version: "encryptionkey@0"
I20260619 12:24:52.329694 13744 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/wal/instance:
uuid: "86026bdd073e435980fb26cf963f9f1b"
format_stamp: "Formatted at 2026-06-19 12:24:52 on dist-test-slave-kr9h"
server_key: "ef12de90260669570b8fbd9faea1f8aa"
server_key_iv: "c3cd63294fdbf44a79a6f37dd8101d5d"
server_key_version: "encryptionkey@0"
I20260619 12:24:52.333531 13744 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 12:24:52.336280 13837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:52.337788 13744 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:52.337927 13744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/wal
uuid: "86026bdd073e435980fb26cf963f9f1b"
format_stamp: "Formatted at 2026-06-19 12:24:52 on dist-test-slave-kr9h"
server_key: "ef12de90260669570b8fbd9faea1f8aa"
server_key_iv: "c3cd63294fdbf44a79a6f37dd8101d5d"
server_key_version: "encryptionkey@0"
I20260619 12:24:52.338048 13744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:52.356695 13744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:52.360363 13744 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:52.360563 13744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:52.367846 13744 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.252:33277
I20260619 12:24:52.368247 13889 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.252:33277 every 8 connection(s)
I20260619 12:24:52.368875 13744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/data/info.pb
I20260619 12:24:52.371655 13890 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:52.374222 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 13744
I20260619 12:24:52.376127 13890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.252@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871890, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.231.252@KRBTEST.COM
I20260619 12:24:52.384886 13363 heartbeater.cc:503] Master 127.6.231.254:45495 was elected leader, sending a full tablet report...
I20260619 12:24:52.388818 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.378217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59730 (local address 127.6.231.254:45495)
0619 12:24:52.378351 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.378356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.378685 (+ 329us) server_negotiation.cc:407] Connection header received
0619 12:24:52.378876 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.378878 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.378923 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.378986 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.379859 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.380517 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.381774 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.382061 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.382183 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.384670 (+ 2487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.384674 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.384676 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.384701 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.386597 (+ 1896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.387105 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.387110 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.387112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.387199 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.387603 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.387605 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.387607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.387844 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.387936 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.388520 (+ 584us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.388661 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 12:24:52.389698 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.378045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:59730)
0619 12:24:52.378563 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.378599 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.378739 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.379060 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.379065 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.379529 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.379705 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.379717 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.380670 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.380673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.381659 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.381669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.382287 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.382293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.382414 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.382971 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.382986 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.384504 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.386705 (+ 2201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.386712 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.386719 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.387002 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.387296 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.387318 (+ 22us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.387323 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.387506 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.387946 (+ 440us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.387952 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.388252 (+ 300us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.388452 (+ 200us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.388794 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":95,"threads_started":1}
I20260619 12:24:52.389699 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.375961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39706 (local address 127.6.231.252:33277)
0619 12:24:52.376392 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.376410 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.376452 (+ 42us) server_negotiation.cc:407] Connection header received
0619 12:24:52.377246 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.377267 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.377678 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.378026 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.378845 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.380159 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.381082 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.381801 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.381914 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.385186 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.385193 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.385205 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.385229 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.387202 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.387699 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.387704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.387706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.387809 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.388105 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.388108 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.388109 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.388331 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.388520 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.388744 (+ 224us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.389019 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":121,"threads_started":1}
I20260619 12:24:52.391615 13890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.252@KRBTEST.COM for oryx/127.6.231.253@KRBTEST.COM
I20260619 12:24:52.399694 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.392091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:38667 (local address 127.0.0.1:52010)
0619 12:24:52.392415 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.392427 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.392484 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.392682 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.392684 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.392831 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.392921 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.392925 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.393854 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.393856 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.394410 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.394417 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.394839 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.394842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.394909 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.395313 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.395324 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.396564 (+ 1240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.398351 (+ 1787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.398354 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.398355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.398545 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.398834 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.398837 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.398839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.398939 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.399293 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.399295 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.399374 (+ 79us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.399465 (+ 91us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.399562 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266}
I20260619 12:24:52.399777 13894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.392222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52010 (local address 127.6.231.253:38667)
0619 12:24:52.392459 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.392463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.392475 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:24:52.392546 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.392548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.392586 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.392656 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.393015 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.393739 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.394517 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.394733 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.394827 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.396694 (+ 1867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.396697 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.396699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.396720 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.398238 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.398633 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.398636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.398637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.398712 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.399018 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.399020 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.399021 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.399191 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.399268 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.399518 (+ 250us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.399612 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":84,"threads_started":1}
I20260619 12:24:52.400453 13890 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.252@KRBTEST.COM for oryx/127.6.231.252@KRBTEST.COM
I20260619 12:24:52.408420 13893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.400923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33277 (local address 127.0.0.1:39712)
0619 12:24:52.401025 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.401039 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.401107 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.401514 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.401517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.401644 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.401703 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.401708 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.402473 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.402479 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.402969 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.402975 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.403413 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.403417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.403486 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.403967 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.403977 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.405340 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.407053 (+ 1713us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.407056 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.407057 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.407299 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.407572 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.407574 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.407575 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.407646 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.407997 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.407999 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.408073 (+ 74us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.408150 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.408298 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260619 12:24:52.408557 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.401081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39712 (local address 127.6.231.252:33277)
0619 12:24:52.401207 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.401210 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.401220 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:24:52.401245 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.401247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.401284 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.401386 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.401832 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.402367 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.403091 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.403315 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.403419 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.405532 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.405535 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.405537 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.405556 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.406927 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.407404 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.407407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.407408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.407485 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.407736 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.407739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.407740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.407916 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.407996 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.408211 (+ 215us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.408358 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":41}
I20260619 12:24:52.411890 13890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: Bootstrap starting.
I20260619 12:24:52.414397 13890 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:52.415179 13890 log.cc:824] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:52.417132 13890 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: No bootstrap required, opened a new log
I20260619 12:24:52.420133 13890 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } }
I20260619 12:24:52.420370 13890 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:52.420450 13890 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86026bdd073e435980fb26cf963f9f1b, State: Initialized, Role: FOLLOWER
I20260619 12:24:52.420933 13890 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } }
I20260619 12:24:52.421511 13896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } } }
I20260619 12:24:52.421653 13896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:24:52.422259 13890 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: configured and running, proceeding with master startup.
W20260619 12:24:52.427101 13907 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:24:52.427205 13907 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:24:52.430655 13744 master_runner.cc:428] Detected that this master 86026bdd073e435980fb26cf963f9f1b is joining an existing cluster
I20260619 12:24:52.430733 13744 master_runner.cc:432] Initiating AddMaster RPC to add 127.6.231.252:33277
I20260619 12:24:52.443181 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 13234
I20260619 12:24:52.444633 13894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.433394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52022 (local address 127.6.231.253:38667)
0619 12:24:52.433564 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.433571 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.433828 (+ 257us) server_negotiation.cc:407] Connection header received
0619 12:24:52.434022 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.434026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.434089 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.434153 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.435087 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.436039 (+ 952us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.437590 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.437976 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.438073 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.440034 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.440037 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.440040 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.440067 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.441829 (+ 1762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.442406 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.442411 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.442413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.442508 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.442828 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.442833 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.442836 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.443086 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.443197 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.444390 (+ 1193us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.444509 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 12:24:52.444675 13914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.433346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:38667 (local address 127.0.0.1:52022)
0619 12:24:52.433732 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.433749 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.433899 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.434325 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.434330 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.434768 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.434897 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.434905 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.436272 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.436275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.437425 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.437462 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.438222 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.438226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.438942 (+ 716us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.439505 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.439518 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.439889 (+ 371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.441980 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.441984 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.441987 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.442257 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.442637 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.442639 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.442641 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.442718 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.443205 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.443208 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.444160 (+ 952us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.444301 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.444459 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"mutex_wait_us":464,"thread_start_us":161,"threads_started":1}
I20260619 12:24:52.449963 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:38201
--local_ip_for_outbound_sockets=127.6.231.193
--tserver_master_addrs=127.6.231.254:45495
--webserver_port=33775
--webserver_interface=127.6.231.193
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab
--principal=oryx/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.231.254:45495,127.6.231.253:38667
--tserver_master_addrs=127.6.231.254:45495,127.6.231.253:38667,127.6.231.252:33277 with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:24:52.462177 13913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.432846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:59746)
0619 12:24:52.433513 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.433530 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.433689 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.434127 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.434130 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.434485 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.434633 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.434642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.436083 (+ 1441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.436086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.437193 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.437204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.438018 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.438022 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.438129 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.438777 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.438793 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.457844 (+ 19051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.460145 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.460150 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.460153 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.460588 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.460918 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.460922 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.460924 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.461007 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.461568 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.461574 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.461687 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.461782 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.461943 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":546,"thread_start_us":132,"threads_started":1}
I20260619 12:24:52.462636 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.433826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39716 (local address 127.6.231.252:33277)
0619 12:24:52.433930 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.433938 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.434377 (+ 439us) server_negotiation.cc:407] Connection header received
0619 12:24:52.434531 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.434534 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.434575 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.434626 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.435189 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.436025 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.438086 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.438551 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.438654 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.459133 (+ 20479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.459138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.459141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.459169 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.460983 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.461418 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.461425 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.461461 (+ 36us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.461537 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.461801 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.461803 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.461805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.461992 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.462057 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.462348 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.462481 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":43}
I20260619 12:24:52.463101 13915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.434019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33277 (local address 127.0.0.1:39716)
0619 12:24:52.434305 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.434318 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.434430 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.434769 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.434771 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.434959 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.435084 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.435091 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.436186 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.436191 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.437970 (+ 1779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.437978 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.438905 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.438911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.439013 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.458438 (+ 19425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.458456 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.458973 (+ 517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.461084 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.461087 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.461091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.461314 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.461630 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.461632 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.461634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.461710 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.462112 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.462114 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.462188 (+ 74us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.462799 (+ 611us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.462940 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":147,"thread_start_us":89,"threads_started":1}
I20260619 12:24:52.463265 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.432973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59746 (local address 127.6.231.254:45495)
0619 12:24:52.433162 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.433238 (+ 76us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.433611 (+ 373us) server_negotiation.cc:407] Connection header received
0619 12:24:52.433814 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.433818 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.433873 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.433951 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.434874 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.435874 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.437342 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.437738 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.437827 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.458057 (+ 20230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.458061 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.458063 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.458093 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.459998 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.460708 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.460712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.460714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.460804 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.461106 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.461109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.461112 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.461384 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.461535 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.462973 (+ 1438us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.463088 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871891, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.254@KRBTEST.COM for oryx/127.6.231.252@KRBTEST.COM
I20260619 12:24:52.476891 13729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.467231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33277 (local address 127.0.0.1:39730)
0619 12:24:52.467404 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.467417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.467489 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.467794 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.467797 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.467987 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.468098 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.468105 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.469271 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.469275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.470182 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.470195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.470873 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.470877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.470976 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.471515 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.471529 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.473116 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.475210 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.475214 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.475217 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.475472 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.475770 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.475774 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.475775 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.475873 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.476343 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.476346 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.476455 (+ 109us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.476563 (+ 108us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.476712 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260619 12:24:52.476945 13892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.467351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39730 (local address 127.6.231.252:33277)
0619 12:24:52.467460 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.467465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.467479 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:24:52.467617 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.467620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.467673 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.467764 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.468252 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.469138 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.470317 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.470742 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.470863 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.473271 (+ 2408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.473275 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.473279 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.473305 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.475091 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.475576 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.475580 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.475582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.475672 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.475971 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.475974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.475976 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.476238 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.476345 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.476637 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.476763 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 12:24:52.478442 13172 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.6.231.252:33277: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "86026bdd073e435980fb26cf963f9f1b"
member_type: NON_VOTER
last_known_addr {
host: "127.6.231.252"
port: 33277
}
attrs {
promote: true
}
}
dest_uuid: "b068dd17324e4c2a8cf783d248709eef"
cas_config_opid_index: 5
I20260619 12:24:52.479110 13172 consensus_queue.cc:237] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } }
I20260619 12:24:52.480305 13731 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 86026bdd073e435980fb26cf963f9f1b (127.6.231.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } } }
I20260619 12:24:52.481921 13731 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.6.231.252:33277
I20260619 12:24:52.482116 13731 sys_catalog.cc:455] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } } }
I20260619 12:24:52.482235 13731 sys_catalog.cc:458] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:52.482685 13744 master.cc:537] Master@127.6.231.252:33277 shutting down...
W20260619 12:24:52.484966 13159 consensus_peers.cc:487] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef -> Peer 86026bdd073e435980fb26cf963f9f1b (127.6.231.252:33277): Couldn't send request to peer 86026bdd073e435980fb26cf963f9f1b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260619 12:24:52.489740 13744 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 0 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:52.490058 13894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.483566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52028 (local address 127.6.231.253:38667)
0619 12:24:52.483701 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.483705 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.484775 (+ 1070us) server_negotiation.cc:407] Connection header received
0619 12:24:52.484981 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.484984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.485047 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.485115 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:52.485604 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.486488 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.488407 (+ 1919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.489107 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.489218 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.489402 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.489818 (+ 416us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.489906 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:24:52.490382 13744 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 0 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:52.490504 13729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.482775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:38667 (local address 127.0.0.1:52028)
0619 12:24:52.484691 (+ 1916us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.484709 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.484856 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.485291 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.485294 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.485314 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:52.485463 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.485471 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.486965 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.486968 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.488240 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.488257 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.489533 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.489536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.489635 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.489689 (+ 54us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.490285 (+ 596us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.490378 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1827}
I20260619 12:24:52.490535 13744 tablet_replica.cc:330] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: stopping tablet replica
I20260619 12:24:52.491062 13782 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 LEARNER]: Refusing update from remote peer b068dd17324e4c2a8cf783d248709eef: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260619 12:24:52.491559 13732 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260619 12:24:52.493727 13782 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 86026bdd073e435980fb26cf963f9f1b (127.6.231.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } } }
I20260619 12:24:52.496138 13811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b068dd17324e4c2a8cf783d248709eef. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } } }
I20260619 12:24:52.496258 13811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: This master's current role is: LEARNER
I20260619 12:24:52.496347 13811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: NON_VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: true } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } } }
I20260619 12:24:52.496420 13811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: This master's current role is: LEARNER
I20260619 12:24:52.497486 13744 master.cc:552] Master@127.6.231.252:33277 shutdown complete.
I20260619 12:24:52.497577 13744 master_runner.cc:305] Clearing existing system tablet
I20260619 12:24:52.497853 13744 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:52.497937 13744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:52.498855 13732 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef: attempting to promote NON_VOTER 7e962ed1f9c84322b912231dd6c821f5 to VOTER
I20260619 12:24:52.499369 13732 consensus_queue.cc:237] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } }
I20260619 12:24:52.500931 13782 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 LEARNER]: Refusing update from remote peer b068dd17324e4c2a8cf783d248709eef: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260619 12:24:52.501010 13729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.500475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33277 (local address 127.0.0.1:39734)
0619 12:24:52.500853 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.500929 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300}
I20260619 12:24:52.501945 13920 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260619 12:24:52.502050 13729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.501813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33277 (local address 127.0.0.1:39736)
0619 12:24:52.501929 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.501986 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 12:24:52.502259 13159 consensus_peers.cc:487] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef -> Peer 86026bdd073e435980fb26cf963f9f1b (127.6.231.252:33277): Couldn't send request to peer 86026bdd073e435980fb26cf963f9f1b. Status: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33277: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 12:24:52.502388 13744 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.001s
I20260619 12:24:52.503329 13921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:52.503505 13744 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:52.503638 13744 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:52.503682 13744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/wal
uuid: "86026bdd073e435980fb26cf963f9f1b"
format_stamp: "Formatted at 2026-06-19 12:24:52 on dist-test-slave-kr9h"
server_key: "ef12de90260669570b8fbd9faea1f8aa"
server_key_iv: "c3cd63294fdbf44a79a6f37dd8101d5d"
server_key_version: "encryptionkey@0"
I20260619 12:24:52.505908 13744 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:52.508495 13744 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260619 12:24:52.508594 13744 log.cc:1196] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260619 12:24:52.508885 13744 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: Deleting consensus metadata
I20260619 12:24:52.509061 13744 master_runner.cc:315] Copying system tablet from 127.6.231.254:45495
I20260619 12:24:52.510322 13744 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: tablet copy: Beginning tablet copy session from remote peer at address 127.6.231.254:45495
I20260619 12:24:52.510995 13920 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7e962ed1f9c84322b912231dd6c821f5 (127.6.231.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } } }
I20260619 12:24:52.511539 13782 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 7e962ed1f9c84322b912231dd6c821f5 (127.6.231.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } } }
I20260619 12:24:52.512594 13920 sys_catalog.cc:455] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } } }
I20260619 12:24:52.512723 13920 sys_catalog.cc:458] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:52.512868 13920 sys_catalog.cc:455] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } } }
I20260619 12:24:52.512940 13920 sys_catalog.cc:458] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:52.513502 13918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } } }
I20260619 12:24:52.513629 13918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:24:52.526182 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.510908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59760 (local address 127.6.231.254:45495)
0619 12:24:52.511145 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.511150 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.514209 (+ 3059us) server_negotiation.cc:407] Connection header received
0619 12:24:52.514403 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.514407 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.514455 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.514505 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.515135 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.515977 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.517262 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.517655 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.517758 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.522304 (+ 4546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.522308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.522312 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.522343 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.524247 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.524771 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.524775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.524777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.524868 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.525174 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.525178 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.525179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.525422 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.525547 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.525873 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.526001 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260619 12:24:52.526633 13927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.513877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:59760)
0619 12:24:52.514117 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.514134 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.514286 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.514661 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.514663 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.514881 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.515007 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.515014 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.516075 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.516080 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.517109 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.517122 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.517797 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.517801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.517910 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.521558 (+ 3648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.521577 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.522082 (+ 505us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.524369 (+ 2287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.524373 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.524376 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.524658 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.524971 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.524974 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.524977 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.525075 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.525535 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.525539 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.525667 (+ 128us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.525810 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.526011 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":76,"threads_started":1}
I20260619 12:24:52.527081 13192 tablet_copy_service.cc:140] P b068dd17324e4c2a8cf783d248709eef: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 86026bdd073e435980fb26cf963f9f1b ({username='oryx', principal='oryx/127.6.231.252@KRBTEST.COM'} at 127.0.0.1:59760)
I20260619 12:24:52.527209 13192 tablet_copy_service.cc:161] P b068dd17324e4c2a8cf783d248709eef: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 86026bdd073e435980fb26cf963f9f1b at {username='oryx', principal='oryx/127.6.231.252@KRBTEST.COM'} at 127.0.0.1:59760: session id = 86026bdd073e435980fb26cf963f9f1b-00000000000000000000000000000000
I20260619 12:24:52.528474 13192 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 12:24:52.529969 13744 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260619 12:24:52.532627 13744 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: tablet copy: Starting download of 0 data blocks...
I20260619 12:24:52.532832 13744 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: tablet copy: Starting download of 1 WAL segments...
I20260619 12:24:52.534541 13744 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 12:24:52.536506 13192 tablet_copy_service.cc:342] P b068dd17324e4c2a8cf783d248709eef: Request end of tablet copy session 86026bdd073e435980fb26cf963f9f1b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.6.231.252@KRBTEST.COM'} at 127.0.0.1:59760
I20260619 12:24:52.536597 13192 tablet_copy_service.cc:433] P b068dd17324e4c2a8cf783d248709eef: ending tablet copy session 86026bdd073e435980fb26cf963f9f1b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 86026bdd073e435980fb26cf963f9f1b
I20260619 12:24:52.544308 13744 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:52.546020 13928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:52.546314 13744 server_base.cc:1034] running on GCE node
W20260619 12:24:52.546450 13929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:52.550027 13931 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:52.550282 13744 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:52.550595 13744 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:52.551760 13744 hybrid_clock.cc:648] HybridClock initialized: now 1781871892551736 us; error 41 us; skew 500 ppm
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:52.553503 13744 init.cc:377] Logged in from keytab as oryx/127.6.231.252@KRBTEST.COM (short username oryx)
I20260619 12:24:52.553995 13744 webserver.cc:460] Webserver started at http://127.6.231.252:38371/ using document root <none> and password file <none>
I20260619 12:24:52.554195 13744 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:52.554241 13744 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:52.555645 13744 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:52.556727 13938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:52.557029 13744 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:52.557108 13744 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/wal
uuid: "86026bdd073e435980fb26cf963f9f1b"
format_stamp: "Formatted at 2026-06-19 12:24:52 on dist-test-slave-kr9h"
server_key: "ef12de90260669570b8fbd9faea1f8aa"
server_key_iv: "c3cd63294fdbf44a79a6f37dd8101d5d"
server_key_version: "encryptionkey@0"
I20260619 12:24:52.557192 13744 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:52.571688 13744 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:52.573984 13744 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:52.579053 13744 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.252:33277
I20260619 12:24:52.579088 13990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.252:33277 every 8 connection(s)
I20260619 12:24:52.579670 13744 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/master-2/data/info.pb
I20260619 12:24:52.581802 13991 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:24:52.582872 13991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: Bootstrap starting.
I20260619 12:24:52.596976 13991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:24:52.597409 13991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: Bootstrap complete.
I20260619 12:24:52.598021 13991 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } }
I20260619 12:24:52.598172 13991 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 86026bdd073e435980fb26cf963f9f1b, State: Initialized, Role: LEARNER
I20260619 12:24:52.598294 13991 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } }
I20260619 12:24:52.598802 13991 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:52.598721 13994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } } }
I20260619 12:24:52.598997 13994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: This master's current role is: LEARNER
I20260619 12:24:52.601812 14005 catalog_manager.cc:1259] Loaded cluster ID: f195b60636ac4e59acfdbbf28c05f478
I20260619 12:24:52.601917 14005 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: loading cluster ID for follower catalog manager: success
I20260619 12:24:52.604130 14005 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: acquiring CA information for follower catalog manager: success
I20260619 12:24:52.605782 14005 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:24:52.607414 13497 heartbeater.cc:503] Master 127.6.231.254:45495 was elected leader, sending a full tablet report...
W20260619 12:24:52.613271 13916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:52.613621 13916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:52.613731 13916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:52.617878 13916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:52.617957 13916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:52.618058 13916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:24:52.623768 13916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab
--principal=oryx/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:38201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=33775
--tserver_master_addrs=127.6.231.254:45495,127.6.231.253:38667,127.6.231.252:33277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:52.624929 13916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:52.626012 13916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:52.635517 14013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:52.635524 14010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:52.635543 14011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:52.635843 13916 server_base.cc:1034] running on GCE node
I20260619 12:24:52.636359 13916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:52.637143 13916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:52.638329 13916 hybrid_clock.cc:648] HybridClock initialized: now 1781871892638295 us; error 48 us; skew 500 ppm
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:52.641191 13916 init.cc:377] Logged in from keytab as oryx/127.6.231.193@KRBTEST.COM (short username oryx)
I20260619 12:24:52.642215 13916 webserver.cc:460] Webserver started at http://127.6.231.193:33775/ using document root <none> and password file <none>
I20260619 12:24:52.642738 13916 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:52.642814 13916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:52.646687 13916 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260619 12:24:52.648733 14020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:52.650133 13916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:24:52.650228 13916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "be0bb06130374b22bbf99f9eb10545ce"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "de9ced78daa5b69f61da7450e9497f47"
server_key_iv: "4c0c794849da42bed936994b39386992"
server_key_version: "encryptionkey@0"
I20260619 12:24:52.650813 13916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:52.671630 13916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:52.675248 13916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:52.675454 13916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:52.676064 13916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:52.676934 13916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:52.676980 13916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:52.677047 13916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:52.677076 13916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:52.687695 13916 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:38201
I20260619 12:24:52.687808 14133 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:38201 every 8 connection(s)
I20260619 12:24:52.689275 13916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:24:52.692108 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 13916
I20260619 12:24:52.692305 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 13368
I20260619 12:24:52.699424 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:43219
--local_ip_for_outbound_sockets=127.6.231.194
--tserver_master_addrs=127.6.231.254:45495
--webserver_port=42567
--webserver_interface=127.6.231.194
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab
--principal=oryx/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.231.254:45495,127.6.231.253:38667
--tserver_master_addrs=127.6.231.254:45495,127.6.231.253:38667,127.6.231.252:33277 with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.252@KRBTEST.COM
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.253@KRBTEST.COM
I20260619 12:24:52.714576 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.698159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:53303 (local address 127.6.231.254:45495)
0619 12:24:52.698380 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.698385 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.699385 (+ 1000us) server_negotiation.cc:407] Connection header received
0619 12:24:52.699923 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.699930 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.700006 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.700067 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.701187 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.702207 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.704361 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.704708 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.704809 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.708499 (+ 3690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.708506 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.708511 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.708548 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.711989 (+ 3441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.712541 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.712544 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.712552 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.712637 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.712933 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.712936 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.712938 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.713145 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.713211 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.714286 (+ 1075us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.714411 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:24:52.714949 14138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.692608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33277 (local address 127.6.231.193:48037)
0619 12:24:52.694092 (+ 1484us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.694153 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.695448 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.699404 (+ 3956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.699416 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.700114 (+ 698us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.700996 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.701017 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.702543 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.702548 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.704090 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.704102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.704858 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.704862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.705110 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.705946 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.705972 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.708615 (+ 2643us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.710936 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.710943 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.710961 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.711436 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.711812 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.711817 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.711819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.712022 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.712638 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.712647 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.713020 (+ 373us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.713321 (+ 301us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.713753 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1234,"mutex_wait_us":252,"spinlock_wait_cycles":3456,"thread_start_us":137,"threads_started":1}
I20260619 12:24:52.715514 14140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.698960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.193:53303)
0619 12:24:52.699285 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.699303 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.699464 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.700241 (+ 777us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.700246 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.700466 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.700996 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.701017 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.702382 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.702385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.704197 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.704206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.704988 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.704991 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.705104 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.706111 (+ 1007us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.706125 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.708256 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.712127 (+ 3871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.712167 (+ 40us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.712170 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.712424 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.712754 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.712757 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.712759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.712836 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.713316 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.713319 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.714051 (+ 732us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.714974 (+ 923us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.715322 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":194,"spinlock_wait_cycles":896,"thread_start_us":87,"threads_started":1}
I20260619 12:24:52.715886 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.698339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:48037 (local address 127.6.231.252:33277)
0619 12:24:52.698728 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.698734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.698749 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:24:52.698798 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.698804 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.698866 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.698959 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.701188 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.702429 (+ 1241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.704243 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.704591 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.704686 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.708762 (+ 4076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.708765 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.708768 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.708795 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.710766 (+ 1971us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.711579 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.711584 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.711588 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.711690 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.712143 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.712148 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.712151 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.712454 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.712561 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.715533 (+ 2972us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.715687 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
I20260619 12:24:52.717844 14135 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45495
I20260619 12:24:52.718083 14135 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:52.718628 14135 heartbeater.cc:511] Master 127.6.231.254:45495 requested a full tablet report, sending...
I20260619 12:24:52.719152 14134 heartbeater.cc:347] Connected to a master server at 127.6.231.252:33277
I20260619 12:24:52.719229 14134 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:52.719396 14134 heartbeater.cc:511] Master 127.6.231.252:33277 requested a full tablet report, sending...
I20260619 12:24:52.719657 13894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.699516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:46055 (local address 127.6.231.253:38667)
0619 12:24:52.699634 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.699638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.701287 (+ 1649us) server_negotiation.cc:407] Connection header received
0619 12:24:52.701504 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.701510 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.701571 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.701639 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.702804 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.703646 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.705065 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.705507 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.705606 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.709162 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.709165 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.709168 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.709195 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.718076 (+ 8881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.718516 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.718520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.718522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.718605 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.718885 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.718888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.718889 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.719060 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.719157 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.719408 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.719513 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:24:52.719678 14141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.699714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:38667 (local address 127.6.231.193:46055)
0619 12:24:52.701190 (+ 1476us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.701207 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.701359 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.701781 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.701785 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.702438 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.702648 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.702659 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.703815 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.703820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.704908 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.704921 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.705817 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.705823 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.705961 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.706423 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.706443 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.708969 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.718203 (+ 9234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.718209 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.718212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.718406 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.718703 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.718706 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.718708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.718788 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.719160 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.719165 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.719253 (+ 88us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.719358 (+ 105us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.719506 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1378,"mutex_wait_us":32,"thread_start_us":80,"threads_started":1}
I20260619 12:24:52.720469 13172 ts_manager.cc:194] Re-registered known tserver with Master: be0bb06130374b22bbf99f9eb10545ce (127.6.231.193:38201)
I20260619 12:24:52.720772 13953 ts_manager.cc:194] Registered new tserver with Master: be0bb06130374b22bbf99f9eb10545ce (127.6.231.193:38201)
I20260619 12:24:52.721197 13172 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:53303
I20260619 12:24:52.722302 14136 heartbeater.cc:347] Connected to a master server at 127.6.231.253:38667
I20260619 12:24:52.722401 14136 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:52.722666 14136 heartbeater.cc:511] Master 127.6.231.253:38667 requested a full tablet report, sending...
I20260619 12:24:52.724255 13772 ts_manager.cc:194] Registered new tserver with Master: be0bb06130374b22bbf99f9eb10545ce (127.6.231.193:38201)
W20260619 12:24:52.831598 14142 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:52.831962 14142 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:52.832026 14142 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:52.836350 14142 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:52.836408 14142 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:52.836472 14142 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:24:52.839411 13631 heartbeater.cc:503] Master 127.6.231.254:45495 was elected leader, sending a full tablet report...
I20260619 12:24:52.842160 14142 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab
--principal=oryx/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:43219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=42567
--tserver_master_addrs=127.6.231.254:45495,127.6.231.253:38667,127.6.231.252:33277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14142
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:52.843282 14142 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:52.844444 14142 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:52.854580 14148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:52.854580 14150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:52.854660 14147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:52.854957 14142 server_base.cc:1034] running on GCE node
I20260619 12:24:52.855626 14142 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:52.856305 14142 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:52.857501 14142 hybrid_clock.cc:648] HybridClock initialized: now 1781871892857469 us; error 49 us; skew 500 ppm
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:52.860598 14142 init.cc:377] Logged in from keytab as oryx/127.6.231.194@KRBTEST.COM (short username oryx)
I20260619 12:24:52.861646 14142 webserver.cc:460] Webserver started at http://127.6.231.194:42567/ using document root <none> and password file <none>
I20260619 12:24:52.862264 14142 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:52.862389 14142 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:52.866797 14142 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:24:52.869292 14157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:52.870726 14142 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:52.870900 14142 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "e3d09f2dbcb2c77f094699ad7a978f9a"
server_key_iv: "d00463240f61316782d51a104052a083"
server_key_version: "encryptionkey@0"
I20260619 12:24:52.871495 14142 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:52.885519 14142 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:52.888593 14142 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:52.888823 14142 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:52.889416 14142 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:52.890331 14142 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:52.890403 14142 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:52.890470 14142 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:52.890522 14142 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:52.899354 14142 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:43219
I20260619 12:24:52.899430 14270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:43219 every 8 connection(s)
I20260619 12:24:52.900382 14142 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:24:52.907099 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 14142
I20260619 12:24:52.907263 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 13502
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.253@KRBTEST.COM
Jun 19 12:24:52 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871892, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.252@KRBTEST.COM
I20260619 12:24:52.915089 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:34249
--local_ip_for_outbound_sockets=127.6.231.195
--tserver_master_addrs=127.6.231.254:45495
--webserver_port=41725
--webserver_interface=127.6.231.195
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab
--principal=oryx/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36043
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.6.231.254:45495,127.6.231.253:38667
--tserver_master_addrs=127.6.231.254:45495,127.6.231.253:38667,127.6.231.252:33277 with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:24:52.918923 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.902286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:35649 (local address 127.6.231.254:45495)
0619 12:24:52.902481 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.902487 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.903682 (+ 1195us) server_negotiation.cc:407] Connection header received
0619 12:24:52.905038 (+ 1356us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.905047 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.905111 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.905216 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.906767 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.907582 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.909625 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.909966 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.910046 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.913922 (+ 3876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.913930 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.913936 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.913973 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.915916 (+ 1943us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.916807 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.916811 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.916812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.916902 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.917596 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.917601 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.917606 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.917843 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.917923 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.918637 (+ 714us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.918758 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 12:24:52.919584 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.904341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:34787 (local address 127.6.231.252:33277)
0619 12:24:52.904462 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.904466 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.904599 (+ 133us) server_negotiation.cc:407] Connection header received
0619 12:24:52.905267 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.905271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.905326 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.905404 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.906770 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.907579 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.909625 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.909981 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.910048 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.915618 (+ 5570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.915625 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.915631 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.915666 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.917578 (+ 1912us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.918098 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.918104 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.918109 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.918191 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.918522 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.918528 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.918533 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.918768 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.918852 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.919217 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.919387 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:24:52.920002 14275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.903023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.194:35649)
0619 12:24:52.903504 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.903546 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.904713 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.905352 (+ 639us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.905354 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.906000 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.906607 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.906623 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.907762 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.907765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.909487 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.909497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.910244 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.910248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.910594 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.911597 (+ 1003us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.911621 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.913715 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.916304 (+ 2589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.916312 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.916331 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.916681 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.917004 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.917009 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.917012 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.917490 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.917986 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.917993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.918322 (+ 329us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.919343 (+ 1021us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.919469 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":406,"spinlock_wait_cycles":23680,"thread_start_us":112,"threads_started":1}
I20260619 12:24:52.920286 14277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.904248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33277 (local address 127.6.231.194:34787)
0619 12:24:52.904518 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.904528 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.904734 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.905455 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.905460 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.906004 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.906612 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.906623 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.907762 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.907765 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.909487 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.909497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.910244 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.910248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.910591 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.911723 (+ 1132us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.911744 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.915440 (+ 3696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.917718 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.917721 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.917724 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.917980 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.918313 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.918319 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.918324 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.918414 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.918912 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.918918 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.919025 (+ 107us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.919791 (+ 766us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.919958 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":443,"spinlock_wait_cycles":30208,"thread_start_us":73,"threads_started":1}
I20260619 12:24:52.920413 13894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.904009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:46703 (local address 127.6.231.253:38667)
0619 12:24:52.904136 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:52.904141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:52.904264 (+ 123us) server_negotiation.cc:407] Connection header received
0619 12:24:52.904924 (+ 660us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:52.904927 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:52.904988 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:52.905098 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:52.906745 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.907551 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.909819 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.910257 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.910369 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.914931 (+ 4562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:52.914934 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:52.914937 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:52.914964 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:52.916837 (+ 1873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.917331 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.917334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.917335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.917423 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:52.917751 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:52.917754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:52.917756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:52.917986 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:52.918066 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:52.920117 (+ 2051us) server_negotiation.cc:299] Negotiation successful
0619 12:24:52.920244 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 12:24:52.920002 14276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:52.903890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:38667 (local address 127.6.231.194:46703)
0619 12:24:52.904178 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:52.904193 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:52.904734 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:52.905293 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:52.905302 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:52.905996 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:52.906607 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.906620 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.908575 (+ 1955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.908581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.909676 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:52.909690 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:52.910526 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:52.910533 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:52.910687 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:52.911757 (+ 1070us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:52.911778 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:52.914762 (+ 2984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:52.916957 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.916960 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.916964 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.917189 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.917559 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:52.917565 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:52.917570 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:52.917654 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:52.918101 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:52.918104 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:52.918322 (+ 218us) client_negotiation.cc:769] Sending connection context
0619 12:24:52.918561 (+ 239us) client_negotiation.cc:240] Negotiation successful
0619 12:24:52.918910 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":122,"spinlock_wait_cycles":3840,"thread_start_us":101,"threads_started":1}
I20260619 12:24:52.921948 14271 heartbeater.cc:347] Connected to a master server at 127.6.231.252:33277
I20260619 12:24:52.922039 14273 heartbeater.cc:347] Connected to a master server at 127.6.231.253:38667
I20260619 12:24:52.922087 14272 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45495
I20260619 12:24:52.922219 14271 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:52.922219 14273 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:52.922885 14273 heartbeater.cc:511] Master 127.6.231.253:38667 requested a full tablet report, sending...
I20260619 12:24:52.922886 14271 heartbeater.cc:511] Master 127.6.231.252:33277 requested a full tablet report, sending...
I20260619 12:24:52.922220 14272 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:52.923625 14272 heartbeater.cc:511] Master 127.6.231.254:45495 requested a full tablet report, sending...
I20260619 12:24:52.924085 13953 ts_manager.cc:194] Registered new tserver with Master: 26bc0b6a6c4a4e089eff44ddf15bcc05 (127.6.231.194:43219)
I20260619 12:24:52.924218 13172 ts_manager.cc:194] Re-registered known tserver with Master: 26bc0b6a6c4a4e089eff44ddf15bcc05 (127.6.231.194:43219)
I20260619 12:24:52.924510 13772 ts_manager.cc:194] Registered new tserver with Master: 26bc0b6a6c4a4e089eff44ddf15bcc05 (127.6.231.194:43219)
I20260619 12:24:52.924907 13172 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:35649
W20260619 12:24:53.032936 14278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:53.033215 14278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:53.033288 14278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:53.037747 14278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:53.037818 14278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:53.037907 14278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:24:53.043473 14278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36043
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab
--principal=oryx/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:34249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=41725
--tserver_master_addrs=127.6.231.254:45495,127.6.231.253:38667,127.6.231.252:33277
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:53.044636 14278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:53.045814 14278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:53.055962 14284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:53.055962 14283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:53.056263 14286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:53.056887 14278 server_base.cc:1034] running on GCE node
I20260619 12:24:53.057384 14278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:53.058218 14278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:53.059406 14278 hybrid_clock.cc:648] HybridClock initialized: now 1781871893059377 us; error 56 us; skew 500 ppm
I20260619 12:24:53.059553 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.053989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33277 (local address 127.0.0.1:39752)
0619 12:24:53.054343 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:53.054362 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:53.054484 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:53.055407 (+ 923us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:53.055413 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:53.055449 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:53.055567 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.055573 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.056778 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.056790 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.058087 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.058103 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.059098 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.059102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.059207 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.059231 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 12:24:53.059305 (+ 74us) client_negotiation.cc:240] Negotiation successful
0619 12:24:53.059390 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":104,"threads_started":1}
I20260619 12:24:53.059564 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.054086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39752 (local address 127.6.231.252:33277)
0619 12:24:53.054460 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.054464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.054480 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:24:53.055022 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.055029 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.055110 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.055278 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:53.055791 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.056644 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.058283 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.058945 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.059059 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.059231 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.059334 (+ 103us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.059440 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260619 12:24:53.060429 13964 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260619 12:24:53.061086 13994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b068dd17324e4c2a8cf783d248709eef. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } } }
I20260619 12:24:53.061254 13994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: This master's current role is: LEARNER
Jun 19 12:24:53 dist-test-slave-kr9h krb5kdc[13127](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871893, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:53.062674 14278 init.cc:377] Logged in from keytab as oryx/127.6.231.195@KRBTEST.COM (short username oryx)
I20260619 12:24:53.063503 14278 webserver.cc:460] Webserver started at http://127.6.231.195:41725/ using document root <none> and password file <none>
I20260619 12:24:53.064031 14278 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:53.064102 14278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:53.068123 14278 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:24:53.070267 14294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:53.071589 14278 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:24:53.071732 14278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "75adfcbbfdae4fbc91f0e4089736c0dd"
format_stamp: "Formatted at 2026-06-19 12:24:51 on dist-test-slave-kr9h"
server_key: "ff5192e777c0bc0f4b1f9554dbf7ef9b"
server_key_iv: "7df7f569a71515116dbc5597da95eb43"
server_key_version: "encryptionkey@0"
I20260619 12:24:53.072275 14278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:53.088840 14278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:53.093149 14278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:53.093380 14278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:53.094115 14278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:24:53.095027 14278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:53.095073 14278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:53.095114 14278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:53.095139 14278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:53.105755 14278 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:34249
I20260619 12:24:53.105790 14407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:34249 every 8 connection(s)
I20260619 12:24:53.106824 14278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:24:53.112756 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 14278
Jun 19 12:24:53 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871893, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.252@KRBTEST.COM
Jun 19 12:24:53 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871893, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.253@KRBTEST.COM
Jun 19 12:24:53 dist-test-slave-kr9h krb5kdc[13127](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871893, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
I20260619 12:24:53.126367 13894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.110584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:56857 (local address 127.6.231.253:38667)
0619 12:24:53.110712 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.110716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.110858 (+ 142us) server_negotiation.cc:407] Connection header received
0619 12:24:53.111795 (+ 937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.111799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.111859 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.111947 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:53.113386 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.114253 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.116359 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.116688 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.116762 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.120672 (+ 3910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:53.120678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:53.120685 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:53.120717 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:53.124084 (+ 3367us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.124828 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.124831 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.124833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.124911 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.125204 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.125207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.125209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.125462 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:53.125526 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.126073 (+ 547us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.126197 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 12:24:53.126367 13211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.108683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:49489 (local address 127.6.231.254:45495)
0619 12:24:53.108813 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.108817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.109874 (+ 1057us) server_negotiation.cc:407] Connection header received
0619 12:24:53.111524 (+ 1650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.111528 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.111580 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.111665 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:53.113385 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.114259 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.116237 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.116561 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.116632 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.121889 (+ 5257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:53.121893 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:53.121896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:53.121924 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:53.123733 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.124435 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.124441 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.124446 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.124541 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.124991 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.124995 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.124996 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.125244 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:53.125313 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.126079 (+ 766us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.126201 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:24:53.126952 14139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.110259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:52131 (local address 127.6.231.252:33277)
0619 12:24:53.110391 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.110397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.110479 (+ 82us) server_negotiation.cc:407] Connection header received
0619 12:24:53.111333 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.111339 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.111405 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.111476 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:53.113702 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.114471 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.116412 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.116772 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.116871 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.121095 (+ 4224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:53.121098 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:53.121101 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:53.121142 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:53.124947 (+ 3805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.125487 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.125490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.125494 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.125577 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.125884 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.125887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.125889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.126097 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:53.126203 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.126697 (+ 494us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.126811 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:24:53.128235 14414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.110491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:38667 (local address 127.6.231.195:56857)
0619 12:24:53.110732 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:53.110744 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:53.111015 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:53.111991 (+ 976us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:53.111994 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:53.112308 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:53.113208 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.113228 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.114439 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.114442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.116246 (+ 1804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.116254 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.116893 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.116896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.117515 (+ 619us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.118249 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:53.118260 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:53.120466 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:53.124473 (+ 4007us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:53.124476 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:53.124478 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:53.124726 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:53.125021 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:53.125025 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:53.125030 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:53.125101 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:53.125588 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:53.125590 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:53.125781 (+ 191us) client_negotiation.cc:769] Sending connection context
0619 12:24:53.126585 (+ 804us) client_negotiation.cc:240] Negotiation successful
0619 12:24:53.127268 (+ 683us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":1026,"spinlock_wait_cycles":9216,"thread_start_us":91,"threads_started":1}
I20260619 12:24:53.128235 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.109175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.195:49489)
0619 12:24:53.109710 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:53.109754 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:53.110990 (+ 1236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:53.111696 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:53.111698 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:53.112309 (+ 611us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:53.113209 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.113228 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.114439 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.114442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.116089 (+ 1647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.116099 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.116833 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.116840 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.117152 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.118198 (+ 1046us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:53.118214 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:53.121738 (+ 3524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:53.123891 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:53.123897 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:53.123917 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:53.124310 (+ 393us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:53.124678 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:53.124682 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:53.124687 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:53.124882 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:53.125402 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:53.125409 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:53.125781 (+ 372us) client_negotiation.cc:769] Sending connection context
0619 12:24:53.126585 (+ 804us) client_negotiation.cc:240] Negotiation successful
0619 12:24:53.127094 (+ 509us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":550,"spinlock_wait_cycles":43648,"thread_start_us":119,"threads_started":1}
I20260619 12:24:53.129133 14413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.110101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33277 (local address 127.6.231.195:52131)
0619 12:24:53.110387 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:53.110402 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:53.110990 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:53.111666 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:53.111676 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:53.112302 (+ 626us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:53.113527 (+ 1225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.113538 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.114624 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.114628 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.116267 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.116281 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.117000 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.117005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.117169 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.117914 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:53.117940 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:53.120946 (+ 3006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:53.125078 (+ 4132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:53.125082 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:53.125084 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:53.125351 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:53.125679 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:53.125682 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:53.125684 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:53.125782 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:53.126250 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:53.126252 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:53.126345 (+ 93us) client_negotiation.cc:769] Sending connection context
0619 12:24:53.127099 (+ 754us) client_negotiation.cc:240] Negotiation successful
0619 12:24:53.128681 (+ 1582us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"mutex_wait_us":1947,"spinlock_wait_cycles":63744,"thread_start_us":103,"threads_started":1}
I20260619 12:24:53.129814 14409 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45495
I20260619 12:24:53.130070 14409 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:53.130100 14408 heartbeater.cc:347] Connected to a master server at 127.6.231.252:33277
I20260619 12:24:53.130167 14408 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:53.130544 14410 heartbeater.cc:347] Connected to a master server at 127.6.231.253:38667
I20260619 12:24:53.130616 14410 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:53.130702 14408 heartbeater.cc:511] Master 127.6.231.252:33277 requested a full tablet report, sending...
I20260619 12:24:53.130774 14410 heartbeater.cc:511] Master 127.6.231.253:38667 requested a full tablet report, sending...
I20260619 12:24:53.131290 14409 heartbeater.cc:511] Master 127.6.231.254:45495 requested a full tablet report, sending...
I20260619 12:24:53.131528 14421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.115713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52030 (local address 127.6.231.253:38667)
0619 12:24:53.117244 (+ 1531us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.117250 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.117263 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:24:53.117299 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.117301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.117350 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.118445 (+ 1095us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:53.118541 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.119342 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.120281 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.120616 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.120738 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.122846 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:53.122851 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:53.122854 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:53.122883 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:53.125094 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.125584 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.125587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.125589 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.125684 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.125981 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.125984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.125986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.126170 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:53.126283 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.130043 (+ 3760us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.131027 (+ 984us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1459,"thread_start_us":78,"threads_started":1}
I20260619 12:24:53.131949 13772 ts_manager.cc:194] Registered new tserver with Master: 75adfcbbfdae4fbc91f0e4089736c0dd (127.6.231.195:34249)
I20260619 12:24:53.131951 13953 ts_manager.cc:194] Registered new tserver with Master: 75adfcbbfdae4fbc91f0e4089736c0dd (127.6.231.195:34249)
I20260619 12:24:53.132839 13172 ts_manager.cc:194] Re-registered known tserver with Master: 75adfcbbfdae4fbc91f0e4089736c0dd (127.6.231.195:34249)
I20260619 12:24:53.133637 14423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.115989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39758 (local address 127.6.231.252:33277)
0619 12:24:53.117268 (+ 1279us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.117272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.118250 (+ 978us) server_negotiation.cc:407] Connection header received
0619 12:24:53.118437 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.118440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.118498 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.118578 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:53.119540 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.120358 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.121628 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.122015 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.122124 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.123907 (+ 1783us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:53.123912 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:53.123918 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:53.123947 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:53.128988 (+ 5041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.129750 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.129754 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.129756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.129837 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.130436 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.130439 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.130440 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.130737 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:53.130936 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.133041 (+ 2105us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.133351 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1205,"thread_start_us":106,"threads_started":1}
I20260619 12:24:53.133800 13172 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:49489
I20260619 12:24:53.136437 14422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.116017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59774 (local address 127.6.231.254:45495)
0619 12:24:53.122696 (+ 6679us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.122702 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.122720 (+ 18us) server_negotiation.cc:407] Connection header received
0619 12:24:53.122761 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.122765 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.124266 (+ 1501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.128316 (+ 4050us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:53.128444 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.129271 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.130411 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.130742 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.131217 (+ 475us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.132379 (+ 1162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:53.132384 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:53.132389 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:53.132423 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:53.134675 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.135111 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.135114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.135116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.135200 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.135511 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.135516 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.135517 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.135771 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:53.135895 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.136116 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.136239 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6586,"thread_start_us":60,"threads_started":1}
I20260619 12:24:53.139300 13172 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59774:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:24:53.142357 13172 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:53.144982 13732 sys_catalog.cc:455] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: NON_VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: true } } }
I20260619 12:24:53.145135 13732 sys_catalog.cc:458] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:53.148598 14427 raft_consensus.cc:1050] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef: attempting to promote NON_VOTER 86026bdd073e435980fb26cf963f9f1b to VOTER
I20260619 12:24:53.148975 14427 consensus_queue.cc:237] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } }
I20260619 12:24:53.150089 13964 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 1 LEARNER]: Refusing update from remote peer b068dd17324e4c2a8cf783d248709eef: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 12:24:53.150496 13732 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [LEADER]: Connected to new peer: Peer: permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260619 12:24:53.150863 13782 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 FOLLOWER]: Refusing update from remote peer b068dd17324e4c2a8cf783d248709eef: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260619 12:24:53.151247 14426 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260619 12:24:53.152755 13732 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 86026bdd073e435980fb26cf963f9f1b (127.6.231.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } } }
I20260619 12:24:53.153872 13782 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 86026bdd073e435980fb26cf963f9f1b (127.6.231.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } } }
I20260619 12:24:53.154311 13732 sys_catalog.cc:455] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } } }
I20260619 12:24:53.154413 13732 sys_catalog.cc:458] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:53.154557 14426 sys_catalog.cc:455] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } } }
I20260619 12:24:53.154699 14426 sys_catalog.cc:458] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:53.155386 13732 sys_catalog.cc:455] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } } }
I20260619 12:24:53.155471 13732 sys_catalog.cc:458] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:53.155345 13964 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 86026bdd073e435980fb26cf963f9f1b (127.6.231.252) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } } }
I20260619 12:24:53.155684 14435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } } }
I20260619 12:24:53.155793 14435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:24:53.157374 13994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b068dd17324e4c2a8cf783d248709eef" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } } }
I20260619 12:24:53.157574 13994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:24:53.165803 14437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.160172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:34249 (local address 127.0.0.1:46430)
0619 12:24:53.160418 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:53.160433 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:53.160525 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:53.161379 (+ 854us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:53.161385 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:53.161403 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:53.161531 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.161542 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.162993 (+ 1451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.162996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.164154 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.164164 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.165270 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.165274 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.165396 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.165414 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 12:24:53.165531 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 12:24:53.165632 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":21,"thread_start_us":109,"threads_started":1}
I20260619 12:24:53.165879 14439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.160606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46430 (local address 127.6.231.195:34249)
0619 12:24:53.160875 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.160887 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.160910 (+ 23us) server_negotiation.cc:407] Connection header received
0619 12:24:53.160983 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.160991 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.161228 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.161702 (+ 474us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:53.161839 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.162889 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.164256 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.165129 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.165415 (+ 286us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.165579 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.165663 (+ 84us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.165758 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":93,"threads_started":1}
I20260619 12:24:53.166401 14438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.160254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:43219 (local address 127.0.0.1:36514)
0619 12:24:53.161074 (+ 820us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:53.161086 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:53.161222 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:53.161720 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:53.161723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:53.161740 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:53.161861 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.161868 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.162820 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.162827 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.164094 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.164104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.165245 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.165248 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.165349 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.166100 (+ 751us) client_negotiation.cc:769] Sending connection context
0619 12:24:53.166170 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 12:24:53.166249 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":765,"mutex_wait_us":1,"thread_start_us":101,"threads_started":1}
I20260619 12:24:53.166482 14440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.160652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36514 (local address 127.6.231.194:43219)
0619 12:24:53.160993 (+ 341us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.161004 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.161198 (+ 194us) server_negotiation.cc:407] Connection header received
0619 12:24:53.161340 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.161346 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.161582 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.161734 (+ 152us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:53.161983 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.162704 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.164211 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.165121 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.165249 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.165421 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.166221 (+ 800us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.166333 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":170,"threads_started":1}
I20260619 12:24:53.166846 14287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.159641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:38201 (local address 127.0.0.1:48036)
0619 12:24:53.159736 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:53.159751 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:53.159830 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:53.161160 (+ 1330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:53.161164 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:53.161181 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:53.161298 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.161308 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.162987 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.162992 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.165230 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.165238 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.166450 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.166454 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.166574 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.166605 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 12:24:53.166655 (+ 50us) client_negotiation.cc:240] Negotiation successful
0619 12:24:53.166735 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":1248}
I20260619 12:24:53.167618 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.159809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48036 (local address 127.6.231.193:38201)
0619 12:24:53.160167 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.160185 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.160211 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:24:53.160300 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.160306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.160979 (+ 673us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.161484 (+ 505us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:53.161655 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.162845 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.165380 (+ 2535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.166304 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.167098 (+ 794us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.167254 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.167309 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.167476 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":93,"threads_started":1}
I20260619 12:24:53.169183 14342 tablet_service.cc:1467] Processing CreateTablet for tablet 3f3b12fcff2449cab2a4ba8450c6997a (DEFAULT_TABLE table=test-table [id=f2d0b0c305df41bdafbb3ee4c0096aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:53.169305 14205 tablet_service.cc:1467] Processing CreateTablet for tablet 3f3b12fcff2449cab2a4ba8450c6997a (DEFAULT_TABLE table=test-table [id=f2d0b0c305df41bdafbb3ee4c0096aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:53.170260 14342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f3b12fcff2449cab2a4ba8450c6997a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:53.170383 14205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f3b12fcff2449cab2a4ba8450c6997a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:53.170426 14068 tablet_service.cc:1467] Processing CreateTablet for tablet 3f3b12fcff2449cab2a4ba8450c6997a (DEFAULT_TABLE table=test-table [id=f2d0b0c305df41bdafbb3ee4c0096aba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:53.171322 14068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f3b12fcff2449cab2a4ba8450c6997a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:53.177235 14441 tablet_bootstrap.cc:492] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd: Bootstrap starting.
I20260619 12:24:53.179001 14443 tablet_bootstrap.cc:492] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce: Bootstrap starting.
I20260619 12:24:53.179036 14442 tablet_bootstrap.cc:492] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05: Bootstrap starting.
I20260619 12:24:53.179981 14441 tablet_bootstrap.cc:654] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:53.180684 14442 tablet_bootstrap.cc:654] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:53.180852 14441 log.cc:824] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:53.181102 14443 tablet_bootstrap.cc:654] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:53.181541 14442 log.cc:824] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:53.182332 14443 log.cc:824] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:53.183112 14441 tablet_bootstrap.cc:492] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd: No bootstrap required, opened a new log
I20260619 12:24:53.183305 14441 ts_tablet_manager.cc:1397] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 12:24:53.183367 14442 tablet_bootstrap.cc:492] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05: No bootstrap required, opened a new log
I20260619 12:24:53.183544 14442 ts_tablet_manager.cc:1397] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 12:24:53.184146 14443 tablet_bootstrap.cc:492] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce: No bootstrap required, opened a new log
I20260619 12:24:53.184316 14443 ts_tablet_manager.cc:1397] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.004s
I20260619 12:24:53.187250 14442 raft_consensus.cc:348] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:53.187501 14442 raft_consensus.cc:374] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:53.187552 14442 raft_consensus.cc:729] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26bc0b6a6c4a4e089eff44ddf15bcc05, State: Initialized, Role: FOLLOWER
I20260619 12:24:53.187736 14441 raft_consensus.cc:348] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:53.187983 14441 raft_consensus.cc:374] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:53.188035 14441 raft_consensus.cc:729] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75adfcbbfdae4fbc91f0e4089736c0dd, State: Initialized, Role: FOLLOWER
I20260619 12:24:53.188026 14442 consensus_queue.cc:260] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:53.188084 14443 raft_consensus.cc:348] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:53.188297 14443 raft_consensus.cc:374] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:53.188357 14443 raft_consensus.cc:729] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be0bb06130374b22bbf99f9eb10545ce, State: Initialized, Role: FOLLOWER
I20260619 12:24:53.188460 14441 consensus_queue.cc:260] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:53.188735 14443 consensus_queue.cc:260] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:53.189075 14442 ts_tablet_manager.cc:1428] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 12:24:53.189364 14441 ts_tablet_manager.cc:1428] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd: Time spent starting tablet: real 0.006s user 0.000s sys 0.007s
I20260619 12:24:53.189577 14443 ts_tablet_manager.cc:1428] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260619 12:24:53.189589 14409 heartbeater.cc:503] Master 127.6.231.254:45495 was elected leader, sending a full tablet report...
I20260619 12:24:53.191304 14135 heartbeater.cc:503] Master 127.6.231.254:45495 was elected leader, sending a full tablet report...
W20260619 12:24:53.191570 14137 tablet.cc:2369] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:24:53.194504 14272 heartbeater.cc:503] Master 127.6.231.254:45495 was elected leader, sending a full tablet report...
W20260619 12:24:53.359150 14411 tablet.cc:2369] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:24:53.368135 14448 raft_consensus.cc:482] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:53.368358 14448 raft_consensus.cc:504] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:53.369565 14448 leader_election.cc:288] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be0bb06130374b22bbf99f9eb10545ce (127.6.231.193:38201), 26bc0b6a6c4a4e089eff44ddf15bcc05 (127.6.231.194:43219)
I20260619 12:24:53.373487 14440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.370059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:45511 (local address 127.6.231.194:43219)
0619 12:24:53.370195 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.370200 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.370216 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:24:53.370277 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.370279 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.370318 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.370408 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:53.370702 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.371352 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.372441 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.372851 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.372944 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.373027 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.373241 (+ 214us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.373323 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:24:53.373528 14436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.369909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:54445 (local address 127.6.231.193:38201)
0619 12:24:53.370041 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.370045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.370506 (+ 461us) server_negotiation.cc:407] Connection header received
0619 12:24:53.370565 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.370568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.370623 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.370720 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:53.370972 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.371483 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.372491 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.372886 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.373027 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.373089 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.373323 (+ 234us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.373407 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 12:24:53.373510 14413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.369831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:43219 (local address 127.6.231.195:45511)
0619 12:24:53.369960 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:53.369978 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:53.370077 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:53.370420 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:53.370423 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:53.370438 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:53.370568 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.370574 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.371470 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.371473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.372305 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.372314 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.372977 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.372984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.373077 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.373101 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 12:24:53.373223 (+ 122us) client_negotiation.cc:240] Negotiation successful
0619 12:24:53.373312 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260619 12:24:53.373513 14412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.369838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:38201 (local address 127.6.231.195:54445)
0619 12:24:53.370039 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:53.370054 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:53.370153 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:53.370733 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:53.370736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:53.370753 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:53.370854 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.370860 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.371618 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.371620 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.372322 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:53.372330 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.373037 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.373043 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:53.373182 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.373213 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 12:24:53.373294 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 12:24:53.373355 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":43}
I20260619 12:24:53.374199 14225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f3b12fcff2449cab2a4ba8450c6997a" candidate_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" is_pre_election: true
I20260619 12:24:53.374405 14088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f3b12fcff2449cab2a4ba8450c6997a" candidate_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be0bb06130374b22bbf99f9eb10545ce" is_pre_election: true
I20260619 12:24:53.374516 14225 raft_consensus.cc:2454] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75adfcbbfdae4fbc91f0e4089736c0dd in term 0.
I20260619 12:24:53.374713 14088 raft_consensus.cc:2454] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 75adfcbbfdae4fbc91f0e4089736c0dd in term 0.
I20260619 12:24:53.375052 14295 leader_election.cc:302] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26bc0b6a6c4a4e089eff44ddf15bcc05, 75adfcbbfdae4fbc91f0e4089736c0dd; no voters:
I20260619 12:24:53.375326 14448 raft_consensus.cc:2781] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:53.375424 14448 raft_consensus.cc:482] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:53.375494 14448 raft_consensus.cc:3037] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:53.376720 14448 raft_consensus.cc:504] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:53.377071 14448 leader_election.cc:288] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [CANDIDATE]: Term 1 election: Requested vote from peers be0bb06130374b22bbf99f9eb10545ce (127.6.231.193:38201), 26bc0b6a6c4a4e089eff44ddf15bcc05 (127.6.231.194:43219)
I20260619 12:24:53.377451 14088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f3b12fcff2449cab2a4ba8450c6997a" candidate_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be0bb06130374b22bbf99f9eb10545ce"
I20260619 12:24:53.377581 14088 raft_consensus.cc:3037] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:53.377537 14225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f3b12fcff2449cab2a4ba8450c6997a" candidate_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05"
I20260619 12:24:53.377733 14225 raft_consensus.cc:3037] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:53.378629 14088 raft_consensus.cc:2454] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75adfcbbfdae4fbc91f0e4089736c0dd in term 1.
I20260619 12:24:53.378809 14225 raft_consensus.cc:2454] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 75adfcbbfdae4fbc91f0e4089736c0dd in term 1.
I20260619 12:24:53.378923 14296 leader_election.cc:302] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 75adfcbbfdae4fbc91f0e4089736c0dd, be0bb06130374b22bbf99f9eb10545ce; no voters:
I20260619 12:24:53.379114 14448 raft_consensus.cc:2781] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:53.379401 14448 raft_consensus.cc:686] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [term 1 LEADER]: Becoming Leader. State: Replica: 75adfcbbfdae4fbc91f0e4089736c0dd, State: Running, Role: LEADER
I20260619 12:24:53.379918 14448 consensus_queue.cc:237] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:53.383601 13171 catalog_manager.cc:5495] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd reported cstate change: term changed from 0 to 1, leader changed from <none> to 75adfcbbfdae4fbc91f0e4089736c0dd (127.6.231.195). New cstate: current_term: 1 leader_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } health_report { overall_health: HEALTHY } } }
I20260619 12:24:53.397411 14439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.393932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46440 (local address 127.6.231.195:34249)
0619 12:24:53.394050 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.394055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.394068 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:24:53.394263 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.394266 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.394315 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.394424 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:53.394670 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.395184 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.396030 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.396250 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.396361 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.396649 (+ 288us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:53.397042 (+ 393us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:53.397185 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.397218 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.397269 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
W20260619 12:24:53.402210 14274 tablet.cc:2369] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:24:53.403877 14225 raft_consensus.cc:1261] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 1 FOLLOWER]: Refusing update from remote peer 75adfcbbfdae4fbc91f0e4089736c0dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:53.403927 14088 raft_consensus.cc:1261] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce [term 1 FOLLOWER]: Refusing update from remote peer 75adfcbbfdae4fbc91f0e4089736c0dd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:53.404633 14448 consensus_queue.cc:1035] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:53.404837 14450 consensus_queue.cc:1035] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [LEADER]: Connected to new peer: Peer: permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:53.425362 13171 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:53.425563 13171 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59774:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:53.430426 13171 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b068dd17324e4c2a8cf783d248709eef: Sending DeleteTablet for 3 replicas of tablet 3f3b12fcff2449cab2a4ba8450c6997a
I20260619 12:24:53.431156 14068 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f3b12fcff2449cab2a4ba8450c6997a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:53 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:48036
I20260619 12:24:53.431192 14205 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f3b12fcff2449cab2a4ba8450c6997a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:53 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:36514
I20260619 12:24:53.431629 14463 tablet_replica.cc:330] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05: stopping tablet replica
I20260619 12:24:53.431640 14464 tablet_replica.cc:330] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce: stopping tablet replica
I20260619 12:24:53.431951 14464 raft_consensus.cc:2229] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:53.431979 14463 raft_consensus.cc:2229] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:53.432291 14464 raft_consensus.cc:2258] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:53.432531 14463 raft_consensus.cc:2258] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:53.433521 14342 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f3b12fcff2449cab2a4ba8450c6997a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:53 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:46430
I20260619 12:24:53.434425 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 13140
I20260619 12:24:53.435350 14463 ts_tablet_manager.cc:1905] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:53.438992 14463 ts_tablet_manager.cc:1918] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:53.439191 14463 log.cc:1196] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/3f3b12fcff2449cab2a4ba8450c6997a
I20260619 12:24:53.439595 14463 ts_tablet_manager.cc:1939] T 3f3b12fcff2449cab2a4ba8450c6997a P 26bc0b6a6c4a4e089eff44ddf15bcc05: Deleting consensus metadata
I20260619 12:24:53.441668 14466 tablet_replica.cc:330] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd: stopping tablet replica
I20260619 12:24:53.442010 14466 raft_consensus.cc:2229] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:24:53.442446 14466 raft_consensus.cc:2258] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 12:24:53.442619 14160 connection.cc:535] server connection from 127.0.0.1:36514 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 12:24:53.444205 14464 ts_tablet_manager.cc:1905] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:53.444412 14421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.433214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52046 (local address 127.6.231.253:38667)
0619 12:24:53.433533 (+ 319us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.433538 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.433550 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:24:53.433586 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.433589 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.433643 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.433719 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:53.435919 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.436780 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.438059 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.438421 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.438495 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.440446 (+ 1951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:53.440450 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:53.440453 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:53.440480 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:53.442440 (+ 1960us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.443039 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.443042 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.443044 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.443138 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.443462 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.443465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.443467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.443683 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:53.443809 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.444130 (+ 321us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.444268 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":244}
I20260619 12:24:53.444661 14466 ts_tablet_manager.cc:1905] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:53.447115 14464 ts_tablet_manager.cc:1918] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:53.447211 14464 log.cc:1196] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-0/wal/wals/3f3b12fcff2449cab2a4ba8450c6997a
I20260619 12:24:53.447520 14464 ts_tablet_manager.cc:1939] T 3f3b12fcff2449cab2a4ba8450c6997a P be0bb06130374b22bbf99f9eb10545ce: Deleting consensus metadata
I20260619 12:24:53.448005 14466 ts_tablet_manager.cc:1918] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:53.448110 14466 log.cc:1196] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/3f3b12fcff2449cab2a4ba8450c6997a
I20260619 12:24:53.448436 14466 ts_tablet_manager.cc:1939] T 3f3b12fcff2449cab2a4ba8450c6997a P 75adfcbbfdae4fbc91f0e4089736c0dd: Deleting consensus metadata
W20260619 12:24:53.449318 14024 connection.cc:535] server connection from 127.0.0.1:48036 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260619 12:24:53.449366 14298 connection.cc:441] server connection from 127.0.0.1:46430 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:46430 (request call id 1) could send its response
I20260619 12:24:53.454511 14423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.433267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39764 (local address 127.6.231.252:33277)
0619 12:24:53.433388 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.433393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.445359 (+ 11966us) server_negotiation.cc:407] Connection header received
0619 12:24:53.445590 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.445594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.445651 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.445714 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:53.446607 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.447395 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.448738 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.449080 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.449174 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.451135 (+ 1961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:53.451139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:53.451142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:53.451167 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:53.452760 (+ 1593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.453160 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.453172 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.453174 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.453257 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.453579 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.453582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.453584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.453802 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:53.453937 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.454199 (+ 262us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.454331 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260619 12:24:53.464278 14421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.454234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52052 (local address 127.6.231.253:38667)
0619 12:24:53.454356 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.454362 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.455002 (+ 640us) server_negotiation.cc:407] Connection header received
0619 12:24:53.455182 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.455185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.455235 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.455288 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:53.456623 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.457495 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.458679 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.459020 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.459135 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.460890 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:53.460893 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:53.460896 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:53.460922 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:53.462682 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.463081 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.463086 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.463087 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.463173 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.463445 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.463448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.463450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.463624 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:53.463755 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.463998 (+ 243us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.464109 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:24:53.464637 14423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:53.455033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39778 (local address 127.6.231.252:33277)
0619 12:24:53.455153 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:53.455157 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:53.455564 (+ 407us) server_negotiation.cc:407] Connection header received
0619 12:24:53.455722 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:53.455725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:53.455767 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:53.455847 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:53.457496 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.458301 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.459315 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:53.459663 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:53.459764 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:53.461222 (+ 1458us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:53.461226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:53.461229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:53.461252 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:53.462896 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.463359 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.463362 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.463364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.463450 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:53.463727 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:53.463730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:53.463732 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:53.463915 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:53.464020 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:53.464384 (+ 364us) server_negotiation.cc:299] Negotiation successful
0619 12:24:53.464487 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:24:54.418869 14473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.418437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.193:49649)
0619 12:24:54.418693 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:54.418778 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":71,"threads_started":1}
W20260619 12:24:54.419255 14135 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:45495 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
I20260619 12:24:54.424609 14474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.424140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.195:56173)
0619 12:24:54.424417 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:54.424517 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":73,"threads_started":1}
I20260619 12:24:54.424860 14475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.424466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.194:47879)
0619 12:24:54.424688 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:54.424775 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":79,"threads_started":1}
W20260619 12:24:54.425014 14409 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:45495 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
W20260619 12:24:54.425192 14272 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:45495 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
I20260619 12:24:54.930436 14476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b068dd17324e4c2a8cf783d248709eef)
I20260619 12:24:54.930512 14477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader b068dd17324e4c2a8cf783d248709eef)
I20260619 12:24:54.930720 14477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } }
I20260619 12:24:54.930686 14476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } }
I20260619 12:24:54.931654 14476 leader_election.cc:288] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b068dd17324e4c2a8cf783d248709eef (127.6.231.254:45495), 86026bdd073e435980fb26cf963f9f1b (127.6.231.252:33277)
I20260619 12:24:54.931664 14477 leader_election.cc:288] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers b068dd17324e4c2a8cf783d248709eef (127.6.231.254:45495), 7e962ed1f9c84322b912231dd6c821f5 (127.6.231.253:38667)
I20260619 12:24:54.932199 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.931768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60266)
0619 12:24:54.932018 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:54.932100 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":80,"threads_started":1}
W20260619 12:24:54.932366 13759 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
I20260619 12:24:54.932911 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.931812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60264)
0619 12:24:54.932728 (+ 916us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:54.932806 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":845,"thread_start_us":79,"threads_started":1}
W20260619 12:24:54.933193 13942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
I20260619 12:24:54.933259 14478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.933028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60276)
0619 12:24:54.933140 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:54.933187 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 12:24:54.933526 13759 leader_election.cc:334] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b068dd17324e4c2a8cf783d248709eef (127.6.231.254:45495): Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
I20260619 12:24:54.934047 14479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.933782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60278)
0619 12:24:54.933888 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:54.933963 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 12:24:54.934624 13942 leader_election.cc:334] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer b068dd17324e4c2a8cf783d248709eef (127.6.231.254:45495): Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
I20260619 12:24:54.936596 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.932108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33277 (local address 127.0.0.1:39790)
0619 12:24:54.932291 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:54.932305 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:54.932507 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:54.932882 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:54.932885 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:54.932905 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:54.933029 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:54.933036 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:54.934243 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:54.934247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:54.935397 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:54.935407 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:54.936233 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:54.936235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:54.936319 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:54.936338 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 12:24:54.936396 (+ 58us) client_negotiation.cc:240] Negotiation successful
0619 12:24:54.936497 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":1,"thread_start_us":60,"threads_started":1}
I20260619 12:24:54.936633 14482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.932354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39790 (local address 127.6.231.252:33277)
0619 12:24:54.932637 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:54.932641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:54.932654 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:24:54.932699 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:54.932702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:54.932750 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:54.933135 (+ 385us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:54.933282 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:54.934086 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:54.935510 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:54.936130 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:54.936236 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:54.936296 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:54.936444 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 12:24:54.936508 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":218,"spinlock_wait_cycles":13312,"thread_start_us":91,"threads_started":1}
I20260619 12:24:54.937173 13964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e962ed1f9c84322b912231dd6c821f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "86026bdd073e435980fb26cf963f9f1b" is_pre_election: true
I20260619 12:24:54.937311 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.932545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:38667 (local address 127.0.0.1:52056)
0619 12:24:54.932806 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:54.932819 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:54.933180 (+ 361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:54.933800 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:54.933825 (+ 25us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:54.933846 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:54.933927 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:54.933934 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:54.934808 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:54.934811 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:54.935982 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:54.935990 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:54.936948 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:54.936951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:54.937039 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:54.937060 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 12:24:54.937132 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 12:24:54.937196 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"mutex_wait_us":18,"spinlock_wait_cycles":5120,"thread_start_us":50,"threads_started":1}
I20260619 12:24:54.937335 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.932173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52056 (local address 127.6.231.253:38667)
0619 12:24:54.932503 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:54.932507 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:54.932878 (+ 371us) server_negotiation.cc:407] Connection header received
0619 12:24:54.933302 (+ 424us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:54.933307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:54.933351 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:54.933410 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:54.934022 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:54.934670 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:54.936142 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:54.936833 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:54.936948 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:54.936997 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:54.937163 (+ 166us) server_negotiation.cc:299] Negotiation successful
0619 12:24:54.937217 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":222,"thread_start_us":60,"threads_started":1}
I20260619 12:24:54.937382 13964 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e962ed1f9c84322b912231dd6c821f5 in term 1.
I20260619 12:24:54.937768 13759 leader_election.cc:302] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e962ed1f9c84322b912231dd6c821f5, 86026bdd073e435980fb26cf963f9f1b; no voters: b068dd17324e4c2a8cf783d248709eef
I20260619 12:24:54.937870 13782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "86026bdd073e435980fb26cf963f9f1b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e962ed1f9c84322b912231dd6c821f5" is_pre_election: true
I20260619 12:24:54.937975 14476 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 12:24:54.938052 13782 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86026bdd073e435980fb26cf963f9f1b in term 1.
I20260619 12:24:54.938104 14476 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 FOLLOWER]: Starting leader election (detected failure of leader b068dd17324e4c2a8cf783d248709eef)
I20260619 12:24:54.938153 14476 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:24:54.939137 14476 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } }
I20260619 12:24:54.939419 14476 leader_election.cc:288] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [CANDIDATE]: Term 2 election: Requested vote from peers b068dd17324e4c2a8cf783d248709eef (127.6.231.254:45495), 86026bdd073e435980fb26cf963f9f1b (127.6.231.252:33277)
I20260619 12:24:54.939661 13942 leader_election.cc:302] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e962ed1f9c84322b912231dd6c821f5, 86026bdd073e435980fb26cf963f9f1b; no voters: b068dd17324e4c2a8cf783d248709eef
I20260619 12:24:54.939795 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.939597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60284)
0619 12:24:54.939676 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:54.939717 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
I20260619 12:24:54.939900 14477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 12:24:54.939949 13964 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "7e962ed1f9c84322b912231dd6c821f5" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "86026bdd073e435980fb26cf963f9f1b"
I20260619 12:24:54.939985 14477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 1 FOLLOWER]: Starting leader election (detected failure of leader b068dd17324e4c2a8cf783d248709eef)
I20260619 12:24:54.940030 14477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:24:54.941015 14477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } }
I20260619 12:24:54.941283 13964 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7e962ed1f9c84322b912231dd6c821f5 in current term 2: Already voted for candidate 86026bdd073e435980fb26cf963f9f1b in this term.
I20260619 12:24:54.941421 14477 leader_election.cc:288] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [CANDIDATE]: Term 2 election: Requested vote from peers b068dd17324e4c2a8cf783d248709eef (127.6.231.254:45495), 7e962ed1f9c84322b912231dd6c821f5 (127.6.231.253:38667)
I20260619 12:24:54.941754 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.941534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60296)
0619 12:24:54.941619 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:54.941694 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 12:24:54.941897 13782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "86026bdd073e435980fb26cf963f9f1b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e962ed1f9c84322b912231dd6c821f5"
I20260619 12:24:54.942058 13782 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 86026bdd073e435980fb26cf963f9f1b in current term 2: Already voted for candidate 7e962ed1f9c84322b912231dd6c821f5 in this term.
I20260619 12:24:54.942390 14483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.942188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60298)
0619 12:24:54.942292 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:54.942339 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 12:24:54.942523 13942 leader_election.cc:334] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b068dd17324e4c2a8cf783d248709eef (127.6.231.254:45495): Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
I20260619 12:24:54.942612 14480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:54.942394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60300)
0619 12:24:54.942505 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:54.942554 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 12:24:54.942610 13942 leader_election.cc:302] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 86026bdd073e435980fb26cf963f9f1b; no voters: 7e962ed1f9c84322b912231dd6c821f5, b068dd17324e4c2a8cf783d248709eef
I20260619 12:24:54.942735 14477 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
W20260619 12:24:54.942867 13759 leader_election.cc:334] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer b068dd17324e4c2a8cf783d248709eef (127.6.231.254:45495): Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
I20260619 12:24:54.942940 13759 leader_election.cc:302] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7e962ed1f9c84322b912231dd6c821f5; no voters: 86026bdd073e435980fb26cf963f9f1b, b068dd17324e4c2a8cf783d248709eef
I20260619 12:24:54.943151 14476 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 12:24:55.420267 14486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:55.419845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.193:35369)
0619 12:24:55.420105 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:55.420172 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":73,"threads_started":1}
I20260619 12:24:55.426075 14487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:55.425611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.195:51471)
0619 12:24:55.425856 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:55.425934 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":90,"threads_started":1}
I20260619 12:24:55.426188 14488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:55.425753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.194:47415)
0619 12:24:55.426002 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:55.426069 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":102,"threads_started":1}
I20260619 12:24:56.421412 14489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:56.420990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.193:58243)
0619 12:24:56.421237 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:56.421296 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":78,"threads_started":1}
I20260619 12:24:56.427222 14491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:56.426804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.194:53011)
0619 12:24:56.427056 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:56.427123 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":54,"threads_started":1}
I20260619 12:24:56.427222 14490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:56.426701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.195:55047)
0619 12:24:56.427061 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:56.427123 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":85,"threads_started":1}
I20260619 12:24:57.030117 14492 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:57.030264 14492 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } }
I20260619 12:24:57.030759 14492 leader_election.cc:288] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers b068dd17324e4c2a8cf783d248709eef (127.6.231.254:45495), 7e962ed1f9c84322b912231dd6c821f5 (127.6.231.253:38667)
I20260619 12:24:57.031463 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.031080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60542)
0619 12:24:57.031283 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.031354 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":74,"threads_started":1}
I20260619 12:24:57.031646 13782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "86026bdd073e435980fb26cf963f9f1b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e962ed1f9c84322b912231dd6c821f5" is_pre_election: true
I20260619 12:24:57.031808 13782 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 86026bdd073e435980fb26cf963f9f1b in term 2.
I20260619 12:24:57.032194 13942 leader_election.cc:302] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7e962ed1f9c84322b912231dd6c821f5, 86026bdd073e435980fb26cf963f9f1b; no voters:
I20260619 12:24:57.032301 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.032078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60546)
0619 12:24:57.032199 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.032245 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260619 12:24:57.032351 14492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 2 FOLLOWER]: Leader pre-election won for term 3
W20260619 12:24:57.032403 13942 leader_election.cc:334] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer b068dd17324e4c2a8cf783d248709eef (127.6.231.254:45495): Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
I20260619 12:24:57.032428 14492 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:57.032491 14492 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 2 FOLLOWER]: Advancing to term 3
I20260619 12:24:57.033556 14492 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } }
I20260619 12:24:57.033879 14492 leader_election.cc:288] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [CANDIDATE]: Term 3 election: Requested vote from peers b068dd17324e4c2a8cf783d248709eef (127.6.231.254:45495), 7e962ed1f9c84322b912231dd6c821f5 (127.6.231.253:38667)
I20260619 12:24:57.034192 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.033969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60558)
0619 12:24:57.034055 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.034133 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260619 12:24:57.034375 13782 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "86026bdd073e435980fb26cf963f9f1b" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "7e962ed1f9c84322b912231dd6c821f5"
I20260619 12:24:57.034511 13782 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 2 FOLLOWER]: Advancing to term 3
I20260619 12:24:57.034880 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.034643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60568)
0619 12:24:57.034759 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.034819 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 12:24:57.034974 13942 leader_election.cc:334] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer b068dd17324e4c2a8cf783d248709eef (127.6.231.254:45495): Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
I20260619 12:24:57.035732 13782 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 86026bdd073e435980fb26cf963f9f1b in term 3.
I20260619 12:24:57.036027 13942 leader_election.cc:302] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7e962ed1f9c84322b912231dd6c821f5, 86026bdd073e435980fb26cf963f9f1b; no voters: b068dd17324e4c2a8cf783d248709eef
I20260619 12:24:57.036243 14492 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 3 FOLLOWER]: Leader election won for term 3
I20260619 12:24:57.036455 14492 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [term 3 LEADER]: Becoming Leader. State: Replica: 86026bdd073e435980fb26cf963f9f1b, State: Running, Role: LEADER
I20260619 12:24:57.036723 14492 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } }
I20260619 12:24:57.038008 14495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86026bdd073e435980fb26cf963f9f1b. Latest consensus state: current_term: 3 leader_uuid: "86026bdd073e435980fb26cf963f9f1b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } } }
I20260619 12:24:57.038141 14495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:57.038435 14497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:57.040517 14497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:57.040920 14497 catalog_manager.cc:1259] Loaded cluster ID: f195b60636ac4e59acfdbbf28c05f478
I20260619 12:24:57.040964 14497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:57.041595 14497 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:57.041975 14497 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: Loaded TSK: 0
I20260619 12:24:57.042651 14497 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:24:57.111989 13951 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39778:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:24:57.116667 13951 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:57.120114 13782 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [term 3 FOLLOWER]: Refusing update from remote peer 86026bdd073e435980fb26cf963f9f1b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260619 12:24:57.120600 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.119805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60584)
0619 12:24:57.120452 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.120519 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":558}
I20260619 12:24:57.120630 14495 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260619 12:24:57.121309 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.121081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.0.0.1:60594)
0619 12:24:57.121188 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.121243 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 12:24:57.121466 13942 consensus_peers.cc:487] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b -> Peer b068dd17324e4c2a8cf783d248709eef (127.6.231.254:45495): Couldn't send request to peer b068dd17324e4c2a8cf783d248709eef. Status: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260619 12:24:57.122470 14499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86026bdd073e435980fb26cf963f9f1b. Latest consensus state: current_term: 3 leader_uuid: "86026bdd073e435980fb26cf963f9f1b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } } }
I20260619 12:24:57.122610 14499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:24:57.123695 14495 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "86026bdd073e435980fb26cf963f9f1b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } } }
I20260619 12:24:57.123822 14495 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:57.123993 14499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "86026bdd073e435980fb26cf963f9f1b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "b068dd17324e4c2a8cf783d248709eef" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45495 } } peers { permanent_uuid: "7e962ed1f9c84322b912231dd6c821f5" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 38667 } attrs { promote: false } } peers { permanent_uuid: "86026bdd073e435980fb26cf963f9f1b" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33277 } attrs { promote: false } } }
I20260619 12:24:57.124172 14499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e962ed1f9c84322b912231dd6c821f5 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:24:57.134124 14493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.130115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:38201 (local address 127.0.0.1:48042)
0619 12:24:57.130264 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.130279 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:57.130383 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:57.130797 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:57.130800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:57.130818 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:57.130934 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:57.130940 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.131926 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.131929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:57.132753 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:57.132761 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.133708 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.133712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:57.133802 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:57.133830 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 12:24:57.133876 (+ 46us) client_negotiation.cc:240] Negotiation successful
0619 12:24:57.133984 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":83,"mutex_wait_us":2}
I20260619 12:24:57.134902 14505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.130214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48042 (local address 127.6.231.193:38201)
0619 12:24:57.130526 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:57.130531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:57.130545 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:24:57.130597 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:57.130601 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:57.130662 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:57.131053 (+ 391us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:57.131177 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.131780 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.132881 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.133579 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.134593 (+ 1014us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:57.134655 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:57.134688 (+ 33us) server_negotiation.cc:299] Negotiation successful
0619 12:24:57.134758 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":119,"threads_started":1}
I20260619 12:24:57.135437 14068 tablet_service.cc:1467] Processing CreateTablet for tablet 224409441d0640d6bd6f53c9f9b4b972 (DEFAULT_TABLE table=test-table [id=c9fed257a3d2487ab936a9a24c3bc9f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:57.135591 14507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.130620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:34249 (local address 127.0.0.1:46448)
0619 12:24:57.131297 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.131308 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:57.131419 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:57.131679 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:57.131683 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:57.131700 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:57.131799 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:57.131807 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.132927 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.132933 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:57.134309 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:57.134320 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.135103 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.135107 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:57.135191 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:57.135256 (+ 65us) client_negotiation.cc:769] Sending connection context
0619 12:24:57.135365 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 12:24:57.135444 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"mutex_wait_us":206,"thread_start_us":99,"threads_started":1}
I20260619 12:24:57.135615 14509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.130923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46448 (local address 127.6.231.195:34249)
0619 12:24:57.131183 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:57.131187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:57.131387 (+ 200us) server_negotiation.cc:407] Connection header received
0619 12:24:57.131508 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:57.131511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:57.131562 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:57.131904 (+ 342us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:57.132008 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.132792 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.134418 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.134918 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.135128 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:57.135185 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:57.135405 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 12:24:57.135477 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":88,"threads_started":1}
I20260619 12:24:57.135757 14068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 224409441d0640d6bd6f53c9f9b4b972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:57.137907 14342 tablet_service.cc:1467] Processing CreateTablet for tablet 224409441d0640d6bd6f53c9f9b4b972 (DEFAULT_TABLE table=test-table [id=c9fed257a3d2487ab936a9a24c3bc9f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:57.138183 14342 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 224409441d0640d6bd6f53c9f9b4b972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:57.138264 14510 tablet_bootstrap.cc:492] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce: Bootstrap starting.
I20260619 12:24:57.139106 14510 tablet_bootstrap.cc:654] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:57.140242 14510 tablet_bootstrap.cc:492] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce: No bootstrap required, opened a new log
I20260619 12:24:57.140353 14510 ts_tablet_manager.cc:1397] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:57.140581 14512 tablet_bootstrap.cc:492] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd: Bootstrap starting.
I20260619 12:24:57.140887 14510 raft_consensus.cc:348] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:57.141009 14510 raft_consensus.cc:374] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:57.141047 14510 raft_consensus.cc:729] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be0bb06130374b22bbf99f9eb10545ce, State: Initialized, Role: FOLLOWER
I20260619 12:24:57.141108 14508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.130824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36530 (local address 127.6.231.194:43219)
0619 12:24:57.131640 (+ 816us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:57.131646 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:57.131661 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:24:57.131707 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:57.131710 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:57.131753 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:57.131850 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:57.132101 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.132824 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.134364 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.134959 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.137651 (+ 2692us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:57.137715 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:57.140821 (+ 3106us) server_negotiation.cc:299] Negotiation successful
0619 12:24:57.140928 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":723,"thread_start_us":93,"threads_started":1}
I20260619 12:24:57.141168 14510 consensus_queue.cc:260] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:57.141557 14510 ts_tablet_manager.cc:1428] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260619 12:24:57.141664 14506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.130568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:43219 (local address 127.0.0.1:36530)
0619 12:24:57.131179 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.131193 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:57.131271 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:57.131869 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:57.131883 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:57.131906 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:57.131988 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:57.131993 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.132941 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.132944 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:57.134219 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:57.134229 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.137650 (+ 3421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.137653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:57.140640 (+ 2987us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:57.140664 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 12:24:57.141320 (+ 656us) client_negotiation.cc:240] Negotiation successful
0619 12:24:57.141507 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"mutex_wait_us":89,"thread_start_us":95,"threads_started":1}
I20260619 12:24:57.142477 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.142003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.193:33287)
0619 12:24:57.142314 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.142375 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":100,"threads_started":1}
I20260619 12:24:57.142822 14205 tablet_service.cc:1467] Processing CreateTablet for tablet 224409441d0640d6bd6f53c9f9b4b972 (DEFAULT_TABLE table=test-table [id=c9fed257a3d2487ab936a9a24c3bc9f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:57.142982 14512 tablet_bootstrap.cc:654] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:57.143112 14205 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 224409441d0640d6bd6f53c9f9b4b972. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:57.144198 14512 tablet_bootstrap.cc:492] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd: No bootstrap required, opened a new log
I20260619 12:24:57.144312 14512 ts_tablet_manager.cc:1397] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 12:24:57.144716 14512 raft_consensus.cc:348] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:57.144833 14512 raft_consensus.cc:374] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:57.144881 14512 raft_consensus.cc:729] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75adfcbbfdae4fbc91f0e4089736c0dd, State: Initialized, Role: FOLLOWER
I20260619 12:24:57.145010 14512 consensus_queue.cc:260] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:57.145388 14512 ts_tablet_manager.cc:1428] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:57.146104 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.145696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.195:51607)
0619 12:24:57.145954 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.146015 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":99,"threads_started":1}
I20260619 12:24:57.146819 14134 heartbeater.cc:503] Master 127.6.231.252:33277 was elected leader, sending a full tablet report...
I20260619 12:24:57.146929 14408 heartbeater.cc:503] Master 127.6.231.252:33277 was elected leader, sending a full tablet report...
I20260619 12:24:57.146963 14517 tablet_bootstrap.cc:492] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05: Bootstrap starting.
I20260619 12:24:57.147533 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.143078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.193:57749)
0619 12:24:57.147400 (+ 4322us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.147457 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4248,"mutex_wait_us":3410}
I20260619 12:24:57.147838 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.147576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.195:34659)
0619 12:24:57.147728 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.147776 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 12:24:57.148932 14517 tablet_bootstrap.cc:654] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:57.150104 14517 tablet_bootstrap.cc:492] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05: No bootstrap required, opened a new log
I20260619 12:24:57.150215 14517 ts_tablet_manager.cc:1397] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 12:24:57.150625 14517 raft_consensus.cc:348] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:57.150738 14517 raft_consensus.cc:374] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:57.150797 14517 raft_consensus.cc:729] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26bc0b6a6c4a4e089eff44ddf15bcc05, State: Initialized, Role: FOLLOWER
I20260619 12:24:57.150977 14517 consensus_queue.cc:260] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:57.151342 14517 ts_tablet_manager.cc:1428] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:57.152014 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.151622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.194:39057)
0619 12:24:57.151854 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.151910 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":112,"threads_started":1}
I20260619 12:24:57.152925 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.152704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.194:37265)
0619 12:24:57.152807 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.152862 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260619 12:24:57.153638 14271 heartbeater.cc:503] Master 127.6.231.252:33277 was elected leader, sending a full tablet report...
I20260619 12:24:57.187740 14520 raft_consensus.cc:482] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:57.187894 14520 raft_consensus.cc:504] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:57.189047 14520 leader_election.cc:288] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers be0bb06130374b22bbf99f9eb10545ce (127.6.231.193:38201), 75adfcbbfdae4fbc91f0e4089736c0dd (127.6.231.195:34249)
I20260619 12:24:57.193634 14521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.189265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:38201 (local address 127.6.231.194:46283)
0619 12:24:57.189507 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.189523 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:57.189690 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:57.189974 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:57.189977 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:57.189993 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:57.190114 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:57.190119 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.191048 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.191054 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:57.192546 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:57.192556 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.193183 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.193186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:57.193261 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:57.193281 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 12:24:57.193387 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 12:24:57.193491 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"mutex_wait_us":181}
I20260619 12:24:57.193619 14505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.189371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:46283 (local address 127.6.231.193:38201)
0619 12:24:57.189537 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:57.189542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:57.189606 (+ 64us) server_negotiation.cc:407] Connection header received
0619 12:24:57.189808 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:57.189812 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:57.189863 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:57.189956 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:57.190221 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.190944 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.192675 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.193058 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.193156 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:57.193197 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:57.193406 (+ 209us) server_negotiation.cc:299] Negotiation successful
0619 12:24:57.193501 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 12:24:57.194183 14088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "224409441d0640d6bd6f53c9f9b4b972" candidate_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be0bb06130374b22bbf99f9eb10545ce" is_pre_election: true
I20260619 12:24:57.194307 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.189315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:34249 (local address 127.6.231.194:41911)
0619 12:24:57.189690 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.189701 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:57.189838 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:57.190111 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:57.190114 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:57.190129 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:57.190246 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:57.190251 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.191094 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.191097 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:57.192547 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:57.192556 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.193870 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.193874 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:57.193983 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:57.194002 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 12:24:57.194092 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 12:24:57.194153 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":98,"thread_start_us":157,"threads_started":1}
I20260619 12:24:57.194341 14088 raft_consensus.cc:2454] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26bc0b6a6c4a4e089eff44ddf15bcc05 in term 0.
I20260619 12:24:57.194350 14509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.189425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:41911 (local address 127.6.231.195:34249)
0619 12:24:57.189580 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:57.189586 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:57.189799 (+ 213us) server_negotiation.cc:407] Connection header received
0619 12:24:57.189963 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:57.189971 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:57.190013 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:57.190103 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:57.190357 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.190983 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.193241 (+ 2258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.193742 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.193877 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:57.193935 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:57.194126 (+ 191us) server_negotiation.cc:299] Negotiation successful
0619 12:24:57.194211 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:24:57.194703 14159 leader_election.cc:302] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26bc0b6a6c4a4e089eff44ddf15bcc05, be0bb06130374b22bbf99f9eb10545ce; no voters:
I20260619 12:24:57.194996 14520 raft_consensus.cc:2781] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:57.194968 14362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "224409441d0640d6bd6f53c9f9b4b972" candidate_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" is_pre_election: true
I20260619 12:24:57.195058 14520 raft_consensus.cc:482] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:57.195096 14520 raft_consensus.cc:3037] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:57.195204 14362 raft_consensus.cc:2454] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26bc0b6a6c4a4e089eff44ddf15bcc05 in term 0.
I20260619 12:24:57.197283 14520 raft_consensus.cc:504] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:57.197690 14520 leader_election.cc:288] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [CANDIDATE]: Term 1 election: Requested vote from peers be0bb06130374b22bbf99f9eb10545ce (127.6.231.193:38201), 75adfcbbfdae4fbc91f0e4089736c0dd (127.6.231.195:34249)
I20260619 12:24:57.198124 14362 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "224409441d0640d6bd6f53c9f9b4b972" candidate_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd"
I20260619 12:24:57.198130 14088 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "224409441d0640d6bd6f53c9f9b4b972" candidate_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "be0bb06130374b22bbf99f9eb10545ce"
I20260619 12:24:57.198248 14362 raft_consensus.cc:3037] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:57.198248 14088 raft_consensus.cc:3037] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:57.199445 14362 raft_consensus.cc:2454] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26bc0b6a6c4a4e089eff44ddf15bcc05 in term 1.
I20260619 12:24:57.199445 14088 raft_consensus.cc:2454] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26bc0b6a6c4a4e089eff44ddf15bcc05 in term 1.
I20260619 12:24:57.199707 14159 leader_election.cc:302] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26bc0b6a6c4a4e089eff44ddf15bcc05, be0bb06130374b22bbf99f9eb10545ce; no voters:
I20260619 12:24:57.199899 14520 raft_consensus.cc:2781] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:57.200140 14520 raft_consensus.cc:686] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 1 LEADER]: Becoming Leader. State: Replica: 26bc0b6a6c4a4e089eff44ddf15bcc05, State: Running, Role: LEADER
I20260619 12:24:57.200450 14520 consensus_queue.cc:237] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } }
I20260619 12:24:57.200850 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.200593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.194:40613)
0619 12:24:57.200734 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.200790 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260619 12:24:57.202905 13953 catalog_manager.cc:5495] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26bc0b6a6c4a4e089eff44ddf15bcc05 (127.6.231.194). New cstate: current_term: 1 leader_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "26bc0b6a6c4a4e089eff44ddf15bcc05" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 43219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 } health_report { overall_health: UNKNOWN } } }
I20260619 12:24:57.235320 14508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.231489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36536 (local address 127.6.231.194:43219)
0619 12:24:57.231639 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:57.231645 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:57.231694 (+ 49us) server_negotiation.cc:407] Connection header received
0619 12:24:57.231932 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:57.231934 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:57.231991 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:57.232103 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:57.232343 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.232854 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.233806 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:57.234023 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:57.234137 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:57.234416 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:57.234936 (+ 520us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:57.235076 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:57.235119 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 12:24:57.235174 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 12:24:57.239243 14088 raft_consensus.cc:1261] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce [term 1 FOLLOWER]: Refusing update from remote peer 26bc0b6a6c4a4e089eff44ddf15bcc05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:57.239720 14362 raft_consensus.cc:1261] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd [term 1 FOLLOWER]: Refusing update from remote peer 26bc0b6a6c4a4e089eff44ddf15bcc05: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:57.240337 14520 consensus_queue.cc:1035] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "be0bb06130374b22bbf99f9eb10545ce" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 38201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:57.240619 14523 consensus_queue.cc:1035] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [LEADER]: Connected to new peer: Peer: permanent_uuid: "75adfcbbfdae4fbc91f0e4089736c0dd" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:57.243075 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.242829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.195:33843)
0619 12:24:57.242926 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.242981 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260619 12:24:57.244396 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.244113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.193:53879)
0619 12:24:57.244250 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.244319 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 12:24:57.246296 14526 mvcc.cc:205] Tried to move back new op lower bound from 7298547291086409728 to 7298547290936197120. Current Snapshot: MvccSnapshot[applied={T|T < 7298547291086409728}]
I20260619 12:24:57.247502 14514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.246964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.193:47597)
0619 12:24:57.247369 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.247430 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329}
I20260619 12:24:57.247817 14522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.247012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.194:51381)
0619 12:24:57.247686 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.247748 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":610}
I20260619 12:24:57.248629 14529 mvcc.cc:205] Tried to move back new op lower bound from 7298547291086409728 to 7298547290936197120. Current Snapshot: MvccSnapshot[applied={T|T < 7298547291086409728}]
I20260619 12:24:57.249269 14518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:57.249027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45495 (local address 127.6.231.195:39707)
0619 12:24:57.249143 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.249199 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:45495: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 12:24:57.249856 14527 mvcc.cc:205] Tried to move back new op lower bound from 7298547291086409728 to 7298547290936197120. Current Snapshot: MvccSnapshot[applied={T|T < 7298547291086409728}]
I20260619 12:24:57.254383 13953 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:57.254565 13953 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39778:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:57.258518 13953 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86026bdd073e435980fb26cf963f9f1b: Sending DeleteTablet for 3 replicas of tablet 224409441d0640d6bd6f53c9f9b4b972
I20260619 12:24:57.259159 14068 tablet_service.cc:1514] Processing DeleteTablet for tablet 224409441d0640d6bd6f53c9f9b4b972 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:57 UTC) from {username='oryx', principal='oryx/127.6.231.252@KRBTEST.COM'} at 127.0.0.1:48042
I20260619 12:24:57.259541 14536 tablet_replica.cc:330] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce: stopping tablet replica
I20260619 12:24:57.259675 14536 raft_consensus.cc:2229] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:57.259724 14205 tablet_service.cc:1514] Processing DeleteTablet for tablet 224409441d0640d6bd6f53c9f9b4b972 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:57 UTC) from {username='oryx', principal='oryx/127.6.231.252@KRBTEST.COM'} at 127.0.0.1:36530
I20260619 12:24:57.259809 14536 raft_consensus.cc:2258] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:57.260012 14342 tablet_service.cc:1514] Processing DeleteTablet for tablet 224409441d0640d6bd6f53c9f9b4b972 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:57 UTC) from {username='oryx', principal='oryx/127.6.231.252@KRBTEST.COM'} at 127.0.0.1:46448
I20260619 12:24:57.260345 14537 tablet_replica.cc:330] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05: stopping tablet replica
I20260619 12:24:57.260499 14537 raft_consensus.cc:2229] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:24:57.260612 14536 ts_tablet_manager.cc:1905] T 224409441d0640d6bd6f53c9f9b4b972 P be0bb06130374b22bbf99f9eb10545ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:57.260938 14537 raft_consensus.cc:2258] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:57.261264 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 13916
I20260619 12:24:57.262473 14537 ts_tablet_manager.cc:1905] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:57.265053 14537 ts_tablet_manager.cc:1918] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:57.265197 14537 log.cc:1196] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/224409441d0640d6bd6f53c9f9b4b972
I20260619 12:24:57.265583 14537 ts_tablet_manager.cc:1939] T 224409441d0640d6bd6f53c9f9b4b972 P 26bc0b6a6c4a4e089eff44ddf15bcc05: Deleting consensus metadata
I20260619 12:24:57.265627 14538 tablet_replica.cc:330] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd: stopping tablet replica
I20260619 12:24:57.265789 14538 raft_consensus.cc:2229] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:57.265933 14538 raft_consensus.cc:2258] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:57.266507 13939 catalog_manager.cc:4841] TS 26bc0b6a6c4a4e089eff44ddf15bcc05 (127.6.231.194:43219): tablet 224409441d0640d6bd6f53c9f9b4b972 (table test-table [id=c9fed257a3d2487ab936a9a24c3bc9f5]) successfully deleted
I20260619 12:24:57.266703 14538 ts_tablet_manager.cc:1905] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:57.269845 14538 ts_tablet_manager.cc:1918] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:57.269937 14538 log.cc:1196] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/224409441d0640d6bd6f53c9f9b4b972
I20260619 12:24:57.270213 14538 ts_tablet_manager.cc:1939] T 224409441d0640d6bd6f53c9f9b4b972 P 75adfcbbfdae4fbc91f0e4089736c0dd: Deleting consensus metadata
W20260619 12:24:57.270416 14506 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:24:57.269728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:38201 (local address 127.0.0.1:48046)
0619 12:24:57.269978 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:57.269997 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:57.270087 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:57.270292 (+ 205us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:38201: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":152}
W20260619 12:24:57.270645 13940 catalog_manager.cc:4584] TS be0bb06130374b22bbf99f9eb10545ce (127.6.231.193:38201): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 224409441d0640d6bd6f53c9f9b4b972: Network error: Client connection negotiation failed: client connection to 127.6.231.193:38201: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 12:24:57.270910 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 14142
I20260619 12:24:57.270957 13942 catalog_manager.cc:4841] TS 75adfcbbfdae4fbc91f0e4089736c0dd (127.6.231.195:34249): tablet 224409441d0640d6bd6f53c9f9b4b972 (table test-table [id=c9fed257a3d2487ab936a9a24c3bc9f5]) successfully deleted
I20260619 12:24:57.277253 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 14278
I20260619 12:24:57.283622 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 13636
I20260619 12:24:57.289526 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 13744
2026-06-19T12:24:57Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7553 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:24:57 dist-test-slave-kr9h krb5kdc[14541](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:24:57 dist-test-slave-kr9h krb5kdc[14541](info): set up 2 sockets
Jun 19 12:24:57 dist-test-slave-kr9h krb5kdc[14541](info): commencing operation
krb5kdc: starting...
W20260619 12:24:58.350383 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:24:58 dist-test-slave-kr9h krb5kdc[14541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:24:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:24:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.254.keytab.
Entry for principal oryx/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.254.keytab.
I20260619 12:24:58.477515 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:36725
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.254.keytab
--principal=oryx/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:36725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:58.590473 14554 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:58.590767 14554 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:58.590870 14554 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:58.595171 14554 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:24:58.595240 14554 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260619 12:24:58.595258 14554 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:58.595276 14554 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:58.595293 14554 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:24:58.595332 14554 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:24:58.600869 14554 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:36725
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.254.keytab
--principal=oryx/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:36725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14554
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:58.602015 14554 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:58.603066 14554 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:58.611670 14560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:58.611672 14559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:58.611670 14562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:58.611989 14554 server_base.cc:1034] running on GCE node
I20260619 12:24:58.612519 14554 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:58.613855 14554 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:58.615016 14554 hybrid_clock.cc:648] HybridClock initialized: now 1781871898614997 us; error 32 us; skew 500 ppm
Jun 19 12:24:58 dist-test-slave-kr9h krb5kdc[14541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871898, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:58.617988 14554 init.cc:377] Logged in from keytab as oryx/127.6.231.254@KRBTEST.COM (short username oryx)
I20260619 12:24:58.618836 14554 webserver.cc:460] Webserver started at http://127.6.231.254:45141/ using document root <none> and password file <none>
I20260619 12:24:58.619361 14554 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:58.619419 14554 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:58.619582 14554 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:58.621662 14554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "c69cc20937fc417bb08ff839795597d6"
format_stamp: "Formatted at 2026-06-19 12:24:58 on dist-test-slave-kr9h"
server_key: "097efb728788aaff039581366c7feafd"
server_key_iv: "abc34ac1dd67b2c587b91a54a2060882"
server_key_version: "encryptionkey@0"
I20260619 12:24:58.622177 14554 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "c69cc20937fc417bb08ff839795597d6"
format_stamp: "Formatted at 2026-06-19 12:24:58 on dist-test-slave-kr9h"
server_key: "097efb728788aaff039581366c7feafd"
server_key_iv: "abc34ac1dd67b2c587b91a54a2060882"
server_key_version: "encryptionkey@0"
I20260619 12:24:58.625926 14554 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 12:24:58.628459 14569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:58.629859 14554 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:58.630005 14554 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "c69cc20937fc417bb08ff839795597d6"
format_stamp: "Formatted at 2026-06-19 12:24:58 on dist-test-slave-kr9h"
server_key: "097efb728788aaff039581366c7feafd"
server_key_iv: "abc34ac1dd67b2c587b91a54a2060882"
server_key_version: "encryptionkey@0"
I20260619 12:24:58.630119 14554 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:58.641861 14554 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:58.645279 14554 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:58.645561 14554 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:58.653553 14554 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:36725
I20260619 12:24:58.653584 14631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:36725 every 8 connection(s)
I20260619 12:24:58.654533 14554 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:24:58.657334 14632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:58.663012 14632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6: Bootstrap starting.
I20260619 12:24:58.664316 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 14554
I20260619 12:24:58.664439 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:24:58.664723 7071 external_mini_cluster.cc:1428] Setting key 2354d158ada280d529bfab1c4655c0d7
I20260619 12:24:58.666102 14632 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:58.666896 14632 log.cc:824] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:58.669092 14632 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6: No bootstrap required, opened a new log
Jun 19 12:24:58 dist-test-slave-kr9h krb5kdc[14541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
I20260619 12:24:58.673863 14632 raft_consensus.cc:348] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69cc20937fc417bb08ff839795597d6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36725 } }
I20260619 12:24:58.674156 14632 raft_consensus.cc:374] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:58.674224 14632 raft_consensus.cc:729] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c69cc20937fc417bb08ff839795597d6, State: Initialized, Role: FOLLOWER
I20260619 12:24:58.674733 14632 consensus_queue.cc:260] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69cc20937fc417bb08ff839795597d6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36725 } }
I20260619 12:24:58.674902 14632 raft_consensus.cc:388] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:24:58.674983 14632 raft_consensus.cc:482] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:24:58.675091 14632 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:58.676380 14632 raft_consensus.cc:504] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69cc20937fc417bb08ff839795597d6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36725 } }
I20260619 12:24:58.676762 14632 leader_election.cc:302] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c69cc20937fc417bb08ff839795597d6; no voters:
I20260619 12:24:58.677186 14632 leader_election.cc:288] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:24:58.677259 14637 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:58.677527 14637 raft_consensus.cc:686] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [term 1 LEADER]: Becoming Leader. State: Replica: c69cc20937fc417bb08ff839795597d6, State: Running, Role: LEADER
I20260619 12:24:58.677829 14637 consensus_queue.cc:237] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69cc20937fc417bb08ff839795597d6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36725 } }
I20260619 12:24:58.678544 14632 sys_catalog.cc:564] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:24:58.680294 14635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:58.665928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46452 (local address 127.6.231.254:36725)
0619 12:24:58.666368 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:58.666385 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:58.666422 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:24:58.667123 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:58.667143 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:58.667489 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:58.667774 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:58.668655 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:58.669767 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:58.671000 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:58.671491 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:58.671614 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:58.674743 (+ 3129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:58.674753 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:58.674769 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:58.674804 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:58.676879 (+ 2075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:58.677601 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:58.677610 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:58.677618 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:58.677757 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:58.678084 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:58.678088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:58.678089 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:58.678537 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:58.678832 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:58.679014 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 12:24:58.679322 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":146,"threads_started":1}
I20260619 12:24:58.681715 14639 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c69cc20937fc417bb08ff839795597d6. Latest consensus state: current_term: 1 leader_uuid: "c69cc20937fc417bb08ff839795597d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69cc20937fc417bb08ff839795597d6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36725 } } }
I20260619 12:24:58.681813 14639 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [sys.catalog]: This master's current role is: LEADER
I20260619 12:24:58.681674 14638 sys_catalog.cc:455] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c69cc20937fc417bb08ff839795597d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c69cc20937fc417bb08ff839795597d6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36725 } } }
I20260619 12:24:58.681962 14638 sys_catalog.cc:458] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6 [sys.catalog]: This master's current role is: LEADER
W20260619 12:24:58.684202 14651 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:24:58.684264 14651 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:24:58.684345 14653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:24:58.685031 14653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:24:58.690851 14653 catalog_manager.cc:1347] Generated new cluster ID: 6c8be16239d94998b40bd362ed9c9ad3
I20260619 12:24:58.690922 14653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:24:58.702503 14653 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:24:58.703812 14653 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:24:58.709839 14653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6: Generated new TSK 0
I20260619 12:24:58.710714 14653 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab.
Entry for principal oryx/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab.
I20260619 12:24:58.742908 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36725
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab
--principal=oryx/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:58.854030 14658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:58.854291 14658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:58.854337 14658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:58.858440 14658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:58.858505 14658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:58.858546 14658 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:24:58.858577 14658 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:24:58.863868 14658 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab
--principal=oryx/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.231.254:36725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:58.864929 14658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:58.865937 14658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:58.875463 14666 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:58.875586 14658 server_base.cc:1034] running on GCE node
W20260619 12:24:58.875502 14664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:58.875483 14663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:58.876183 14658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:58.876817 14658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:58.878017 14658 hybrid_clock.cc:648] HybridClock initialized: now 1781871898878001 us; error 26 us; skew 500 ppm
Jun 19 12:24:58 dist-test-slave-kr9h krb5kdc[14541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871898, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:58.881213 14658 init.cc:377] Logged in from keytab as oryx/127.6.231.193@KRBTEST.COM (short username oryx)
I20260619 12:24:58.882097 14658 webserver.cc:460] Webserver started at http://127.6.231.193:35349/ using document root <none> and password file <none>
I20260619 12:24:58.882616 14658 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:58.882676 14658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:58.882838 14658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:58.884825 14658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "1007bb3e7e554133beb50e603f7f7aa6"
format_stamp: "Formatted at 2026-06-19 12:24:58 on dist-test-slave-kr9h"
server_key: "ae37469270bf0e75d250e02d0c780956"
server_key_iv: "3262e33506caf7ba0740f83396bda9f3"
server_key_version: "encryptionkey@0"
I20260619 12:24:58.885352 14658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "1007bb3e7e554133beb50e603f7f7aa6"
format_stamp: "Formatted at 2026-06-19 12:24:58 on dist-test-slave-kr9h"
server_key: "ae37469270bf0e75d250e02d0c780956"
server_key_iv: "3262e33506caf7ba0740f83396bda9f3"
server_key_version: "encryptionkey@0"
I20260619 12:24:58.889084 14658 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:58.891640 14673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:58.892884 14658 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:24:58.893014 14658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "1007bb3e7e554133beb50e603f7f7aa6"
format_stamp: "Formatted at 2026-06-19 12:24:58 on dist-test-slave-kr9h"
server_key: "ae37469270bf0e75d250e02d0c780956"
server_key_iv: "3262e33506caf7ba0740f83396bda9f3"
server_key_version: "encryptionkey@0"
I20260619 12:24:58.893122 14658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:58.902827 14658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:58.906121 14658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:58.906322 14658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:58.907858 14658 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:58.907907 14658 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:58.907946 14658 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:58.907995 14658 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:24:58 dist-test-slave-kr9h krb5kdc[14541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871898, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
I20260619 12:24:58.919235 14658 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:36217
I20260619 12:24:58.919339 14788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:36217 every 8 connection(s)
I20260619 12:24:58.920221 14658 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:24:58.922138 14635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:58.908665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:36575 (local address 127.6.231.254:36725)
0619 12:24:58.908833 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:58.908839 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:58.909700 (+ 861us) server_negotiation.cc:407] Connection header received
0619 12:24:58.910994 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:58.910999 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:58.911068 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:58.911133 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:58.913104 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:58.913875 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:58.915167 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:58.915415 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:58.915527 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:58.918232 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:58.918235 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:58.918238 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:58.918262 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:58.919935 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:58.920534 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:58.920536 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:58.920538 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:58.920616 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:58.921060 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:58.921064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:58.921066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:58.921310 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:58.921395 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:58.921910 (+ 515us) server_negotiation.cc:299] Negotiation successful
0619 12:24:58.922019 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 12:24:58.922962 14682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:58.909025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36725 (local address 127.6.231.193:36575)
0619 12:24:58.909527 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:58.909585 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:58.910715 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:58.911601 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:58.911610 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:58.912099 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:58.912880 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:58.912904 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:58.914003 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:58.914007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:58.915051 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:58.915061 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:58.915579 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:58.915583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:58.915749 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:58.916383 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:58.916399 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:58.918041 (+ 1642us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:58.920049 (+ 2008us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:58.920054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:58.920068 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:58.920414 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:58.920791 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:58.920794 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:58.920797 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:58.920955 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:58.921416 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:58.921421 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:58.921696 (+ 275us) client_negotiation.cc:769] Sending connection context
0619 12:24:58.921852 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 12:24:58.922093 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":108,"threads_started":1}
I20260619 12:24:58.924327 14789 heartbeater.cc:347] Connected to a master server at 127.6.231.254:36725
I20260619 12:24:58.924471 14789 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:58.925055 14789 heartbeater.cc:511] Master 127.6.231.254:36725 requested a full tablet report, sending...
I20260619 12:24:58.926707 14583 ts_manager.cc:194] Registered new tserver with Master: 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193:36217)
I20260619 12:24:58.928042 14583 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:36575
I20260619 12:24:58.929558 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 14658
I20260619 12:24:58.929745 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:24:58.930120 7071 external_mini_cluster.cc:1428] Setting key 841d6cb85a95245ff87aca072652237c
I20260619 12:24:58.934139 14635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:58.926490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:47909 (local address 127.6.231.254:36725)
0619 12:24:58.926615 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:58.926619 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:58.926792 (+ 173us) server_negotiation.cc:407] Connection header received
0619 12:24:58.926951 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:58.926955 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:58.926999 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:58.927045 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:58.927610 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:58.928151 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:58.929267 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:58.929620 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:58.929730 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:58.931005 (+ 1275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:58.931009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:58.931011 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:58.931034 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:58.932430 (+ 1396us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:58.932860 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:58.932865 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:58.932867 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:58.932944 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:58.933233 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:58.933238 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:58.933240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:58.933409 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:58.933539 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:58.933880 (+ 341us) server_negotiation.cc:299] Negotiation successful
0619 12:24:58.933985 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":48}
I20260619 12:24:58.934147 14795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:58.926440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36725 (local address 127.6.231.193:47909)
0619 12:24:58.926701 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:58.926716 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:58.926839 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:58.927174 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:58.927176 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:58.927379 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:58.927496 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:58.927503 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:58.928288 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:58.928292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:58.929139 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:58.929149 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:58.929852 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:58.929856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:58.929961 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:58.930536 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:58.930552 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:58.930877 (+ 325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:58.932545 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:58.932548 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:58.932550 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:58.932759 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:58.933050 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:58.933053 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:58.933055 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:58.933138 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:58.933577 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:58.933579 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:58.933709 (+ 130us) client_negotiation.cc:769] Sending connection context
0619 12:24:58.933816 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 12:24:58.933949 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":98,"threads_started":1}
WARNING: no policy specified for oryx/127.6.231.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.231.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab.
Entry for principal oryx/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab.
I20260619 12:24:58.956547 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36725
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab
--principal=oryx/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:59.069324 14798 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:59.069765 14798 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:59.069845 14798 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:59.074047 14798 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:59.074157 14798 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:59.074237 14798 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:24:59.074326 14798 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:24:59.080353 14798 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab
--principal=oryx/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.231.254:36725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14798
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:59.081562 14798 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:59.082706 14798 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:59.093376 14803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:59.093545 14798 server_base.cc:1034] running on GCE node
W20260619 12:24:59.093375 14804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:59.093370 14806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:59.094274 14798 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:59.095031 14798 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:59.096239 14798 hybrid_clock.cc:648] HybridClock initialized: now 1781871899096216 us; error 37 us; skew 500 ppm
Jun 19 12:24:59 dist-test-slave-kr9h krb5kdc[14541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871899, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:59.099663 14798 init.cc:377] Logged in from keytab as oryx/127.6.231.194@KRBTEST.COM (short username oryx)
I20260619 12:24:59.100737 14798 webserver.cc:460] Webserver started at http://127.6.231.194:35781/ using document root <none> and password file <none>
I20260619 12:24:59.101388 14798 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:59.101511 14798 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:59.101832 14798 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:59.103963 14798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "657d8ebe44b74eb6952d70a0d44fbb25"
format_stamp: "Formatted at 2026-06-19 12:24:59 on dist-test-slave-kr9h"
server_key: "d739b507f940b5f6b77d0986b44ee470"
server_key_iv: "faf9b7b1d99d3b92061c3169dc9dd0bb"
server_key_version: "encryptionkey@0"
I20260619 12:24:59.104552 14798 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "657d8ebe44b74eb6952d70a0d44fbb25"
format_stamp: "Formatted at 2026-06-19 12:24:59 on dist-test-slave-kr9h"
server_key: "d739b507f940b5f6b77d0986b44ee470"
server_key_iv: "faf9b7b1d99d3b92061c3169dc9dd0bb"
server_key_version: "encryptionkey@0"
I20260619 12:24:59.108587 14798 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260619 12:24:59.111362 14813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:59.112874 14798 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:24:59.113044 14798 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "657d8ebe44b74eb6952d70a0d44fbb25"
format_stamp: "Formatted at 2026-06-19 12:24:59 on dist-test-slave-kr9h"
server_key: "d739b507f940b5f6b77d0986b44ee470"
server_key_iv: "faf9b7b1d99d3b92061c3169dc9dd0bb"
server_key_version: "encryptionkey@0"
I20260619 12:24:59.113196 14798 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:59.131539 14798 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:59.135362 14798 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:59.135586 14798 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:59.137137 14798 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:59.137215 14798 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:59.137292 14798 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:59.137342 14798 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:24:59 dist-test-slave-kr9h krb5kdc[14541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871899, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
I20260619 12:24:59.148819 14798 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:35953
I20260619 12:24:59.149010 14928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:35953 every 8 connection(s)
I20260619 12:24:59.149983 14798 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:24:59.152144 14635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.138243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:54227 (local address 127.6.231.254:36725)
0619 12:24:59.138415 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.138420 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.139328 (+ 908us) server_negotiation.cc:407] Connection header received
0619 12:24:59.140447 (+ 1119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.140450 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.140516 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.140617 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:59.142310 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.142906 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.144226 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.144517 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.144635 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.147584 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:59.147587 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:59.147589 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:59.147616 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:59.149600 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:59.150292 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:59.150296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:59.150298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:59.150443 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:59.151019 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:59.151022 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:59.151023 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:59.151255 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:59.151352 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.151900 (+ 548us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.152016 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 12:24:59.152940 14822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.138643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36725 (local address 127.6.231.194:54227)
0619 12:24:59.139144 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:59.139197 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:59.140174 (+ 977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:59.140938 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:59.140945 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:59.141529 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:59.142146 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.142162 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.143042 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.143046 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.144077 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.144096 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.144698 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.144702 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.144908 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.145695 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:59.145712 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:59.147378 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:59.149735 (+ 2357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:59.149742 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:59.149759 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:59.150136 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:59.150740 (+ 604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:59.150744 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:59.150751 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:59.150923 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:59.151373 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:59.151378 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:59.151634 (+ 256us) client_negotiation.cc:769] Sending connection context
0619 12:24:59.151797 (+ 163us) client_negotiation.cc:240] Negotiation successful
0619 12:24:59.152067 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"thread_start_us":144,"threads_started":1}
I20260619 12:24:59.153498 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 14798
I20260619 12:24:59.153630 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:24:59.154007 7071 external_mini_cluster.cc:1428] Setting key fd139f2dd36a9fdc9d5723ac9e64ce5a
I20260619 12:24:59.154497 14929 heartbeater.cc:347] Connected to a master server at 127.6.231.254:36725
I20260619 12:24:59.154633 14929 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:59.155180 14929 heartbeater.cc:511] Master 127.6.231.254:36725 requested a full tablet report, sending...
I20260619 12:24:59.156258 14584 ts_manager.cc:194] Registered new tserver with Master: 657d8ebe44b74eb6952d70a0d44fbb25 (127.6.231.194:35953)
I20260619 12:24:59.156985 14584 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:54227
I20260619 12:24:59.164484 14935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.156793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36725 (local address 127.6.231.194:37607)
0619 12:24:59.157085 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:59.157100 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:59.157198 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:59.157578 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:59.157581 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:59.157783 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:59.157878 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.157890 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.158635 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.158648 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.159209 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.159216 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.159828 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.159831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.159911 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.160455 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:59.160469 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:59.160899 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:59.162980 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:59.162982 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:59.162984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:59.163178 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:59.163464 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:59.163466 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:59.163467 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:59.163540 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:59.164044 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:59.164046 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:59.164121 (+ 75us) client_negotiation.cc:769] Sending connection context
0619 12:24:59.164198 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 12:24:59.164334 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":112,"threads_started":1}
I20260619 12:24:59.164615 14635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.156830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:37607 (local address 127.6.231.254:36725)
0619 12:24:59.156928 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.156932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.157165 (+ 233us) server_negotiation.cc:407] Connection header received
0619 12:24:59.157324 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.157327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.157369 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.157420 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:59.157999 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.158512 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.159324 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.159702 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.159821 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.161041 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:59.161046 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:59.161050 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:59.161079 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:59.162866 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:59.163275 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:59.163280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:59.163282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:59.163371 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:59.163626 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:59.163632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:59.163634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:59.163951 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:59.164059 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.164285 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.164415 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
WARNING: no policy specified for oryx/127.6.231.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.231.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab.
Entry for principal oryx/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab.
I20260619 12:24:59.184185 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36725
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab
--principal=oryx/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34045
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:24:59.298445 14938 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:24:59.298703 14938 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:24:59.298753 14938 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:24:59.302829 14938 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:24:59.302883 14938 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:24:59.302922 14938 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260619 12:24:59.302954 14938 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:24:59.308100 14938 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34045
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab
--principal=oryx/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.6.231.254:36725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14938
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:24:59.309145 14938 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:24:59.310199 14938 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:24:59.319823 14943 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:59.319842 14946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:24:59.319845 14944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:24:59.320161 14938 server_base.cc:1034] running on GCE node
I20260619 12:24:59.320544 14938 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:24:59.321200 14938 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:24:59.322359 14938 hybrid_clock.cc:648] HybridClock initialized: now 1781871899322363 us; error 66 us; skew 500 ppm
Jun 19 12:24:59 dist-test-slave-kr9h krb5kdc[14541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871899, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:24:59.325218 14938 init.cc:377] Logged in from keytab as oryx/127.6.231.195@KRBTEST.COM (short username oryx)
I20260619 12:24:59.326172 14938 webserver.cc:460] Webserver started at http://127.6.231.195:39501/ using document root <none> and password file <none>
I20260619 12:24:59.326776 14938 fs_manager.cc:359] Metadata directory not provided
I20260619 12:24:59.326871 14938 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:24:59.327103 14938 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:24:59.329185 14938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "2c3c56beeca344caa27c3cfe57e57a0b"
format_stamp: "Formatted at 2026-06-19 12:24:59 on dist-test-slave-kr9h"
server_key: "a8d22f550643ae34d33822309551d64d"
server_key_iv: "6c6ee29550cbddb52eff9e9db8ba3ba8"
server_key_version: "encryptionkey@0"
I20260619 12:24:59.329839 14938 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "2c3c56beeca344caa27c3cfe57e57a0b"
format_stamp: "Formatted at 2026-06-19 12:24:59 on dist-test-slave-kr9h"
server_key: "a8d22f550643ae34d33822309551d64d"
server_key_iv: "6c6ee29550cbddb52eff9e9db8ba3ba8"
server_key_version: "encryptionkey@0"
I20260619 12:24:59.333691 14938 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 12:24:59.336262 14953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:59.337672 14938 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:24:59.337852 14938 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "2c3c56beeca344caa27c3cfe57e57a0b"
format_stamp: "Formatted at 2026-06-19 12:24:59 on dist-test-slave-kr9h"
server_key: "a8d22f550643ae34d33822309551d64d"
server_key_iv: "6c6ee29550cbddb52eff9e9db8ba3ba8"
server_key_version: "encryptionkey@0"
I20260619 12:24:59.337975 14938 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:24:59.362238 14938 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:24:59.366263 14938 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:24:59.366518 14938 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:24:59.368198 14938 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:24:59.368280 14938 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:24:59.368348 14938 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:24:59.368393 14938 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Jun 19 12:24:59 dist-test-slave-kr9h krb5kdc[14541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871899, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
I20260619 12:24:59.379415 14938 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:34767
I20260619 12:24:59.379431 15068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:34767 every 8 connection(s)
I20260619 12:24:59.380388 14938 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:24:59.380746 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 14938
I20260619 12:24:59.380887 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:24:59.381220 7071 external_mini_cluster.cc:1428] Setting key 82f8057f2c69841ef912081abf7bfc67
I20260619 12:24:59.383105 14635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.369174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:58039 (local address 127.6.231.254:36725)
0619 12:24:59.369318 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.369321 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.370546 (+ 1225us) server_negotiation.cc:407] Connection header received
0619 12:24:59.371847 (+ 1301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.371850 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.371908 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.372023 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:59.373776 (+ 1753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.374320 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.375723 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.376061 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.376190 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.379032 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:59.379035 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:59.379038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:59.379073 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:59.380792 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:59.381382 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:59.381386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:59.381388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:59.381510 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:59.381907 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:59.381911 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:59.381913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:59.382186 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:59.382287 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.382844 (+ 557us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.382964 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:24:59.383955 14962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.369777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36725 (local address 127.6.231.195:58039)
0619 12:24:59.370343 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:59.370407 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:59.371574 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:59.372236 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:59.372258 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:59.372767 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:59.373584 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.373606 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.374454 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.374458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.375604 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.375615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.376241 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.376245 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.376449 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.377099 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:59.377115 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:59.378843 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:59.380924 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:59.380929 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:59.380943 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:59.381264 (+ 321us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:59.381643 (+ 379us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:59.381646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:59.381648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:59.381781 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:59.382299 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:59.382307 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:59.382555 (+ 248us) client_negotiation.cc:769] Sending connection context
0619 12:24:59.382746 (+ 191us) client_negotiation.cc:240] Negotiation successful
0619 12:24:59.383031 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":138,"threads_started":1}
I20260619 12:24:59.385020 15069 heartbeater.cc:347] Connected to a master server at 127.6.231.254:36725
I20260619 12:24:59.385188 15069 heartbeater.cc:464] Registering TS with master...
I20260619 12:24:59.385746 15069 heartbeater.cc:511] Master 127.6.231.254:36725 requested a full tablet report, sending...
I20260619 12:24:59.386780 14583 ts_manager.cc:194] Registered new tserver with Master: 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195:34767)
I20260619 12:24:59.387713 14583 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:58039
I20260619 12:24:59.395435 14635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.387452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:45965 (local address 127.6.231.254:36725)
0619 12:24:59.387574 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.387578 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.387786 (+ 208us) server_negotiation.cc:407] Connection header received
0619 12:24:59.387928 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.387931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.387979 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.388059 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:59.388577 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.389417 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.390266 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.390482 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.390564 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.391803 (+ 1239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:59.391806 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:59.391808 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:59.391830 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:59.393384 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:59.393918 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:59.393921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:59.393923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:59.394062 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:59.394415 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:59.394418 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:59.394420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:59.394750 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:59.394832 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.395168 (+ 336us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.395276 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:24:59.395438 15075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.387339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36725 (local address 127.6.231.195:45965)
0619 12:24:59.387606 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:59.387703 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:59.387832 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:59.388094 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:59.388097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:59.388340 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:59.388445 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.388451 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.389587 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.389590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.390158 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.390165 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.390655 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.390658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.390733 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.391213 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:59.391230 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:59.391605 (+ 375us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:59.393551 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:59.393553 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:59.393555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:59.393796 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:59.394193 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:59.394197 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:59.394200 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:59.394297 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:59.394855 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:59.394861 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:59.394951 (+ 90us) client_negotiation.cc:769] Sending connection context
0619 12:24:59.395079 (+ 128us) client_negotiation.cc:240] Negotiation successful
0619 12:24:59.395228 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":101,"threads_started":1}
I20260619 12:24:59.395679 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:24:59 dist-test-slave-kr9h krb5kdc[14541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871899, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Jun 19 12:24:59 dist-test-slave-kr9h krb5kdc[14541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:24:59 dist-test-slave-kr9h krb5kdc[14541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 12:24:59.421208 14635 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:24:59.414129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46722 (local address 127.6.231.254:36725)
0619 12:24:59.414298 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.414302 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.414316 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:24:59.414357 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.414360 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.414399 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.414498 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:59.415308 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.415870 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.416929 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.417160 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.417253 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.421016 (+ 3763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46722: BlockingRecv error: recv got EOF from 127.0.0.1:46722 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Jun 19 12:24:59 dist-test-slave-kr9h krb5kdc[14541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871899, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
I20260619 12:24:59.431111 14635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.422826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46732 (local address 127.6.231.254:36725)
0619 12:24:59.422947 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.422954 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.423045 (+ 91us) server_negotiation.cc:407] Connection header received
0619 12:24:59.423202 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.423205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.423261 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.423351 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:59.423961 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.424564 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.425415 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.425652 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.425770 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.428153 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:59.428160 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:59.428165 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:59.428205 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:59.429640 (+ 1435us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:59.430042 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:59.430044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:59.430046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:59.430123 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:59.430354 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:59.430356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:59.430357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:59.430554 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:59.430663 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.430884 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.430983 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:24:59.434469 14583 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:24:59.437666 14583 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:59.452462 15091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.446360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:35953 (local address 127.0.0.1:39614)
0619 12:24:59.447141 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:59.447156 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:59.447227 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:59.448171 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:59.448174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:59.448187 (+ 13us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:59.448250 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.448258 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.449674 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.449679 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.450915 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.450924 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.452002 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.452004 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.452084 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.452115 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 12:24:59.452234 (+ 119us) client_negotiation.cc:240] Negotiation successful
0619 12:24:59.452330 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":708,"mutex_wait_us":58,"thread_start_us":91,"threads_started":1}
I20260619 12:24:59.452534 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.446970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39614 (local address 127.6.231.194:35953)
0619 12:24:59.447741 (+ 771us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.447752 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.447773 (+ 21us) server_negotiation.cc:407] Connection header received
0619 12:24:59.447845 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.447850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.448037 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.448190 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:59.448327 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.449529 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.451034 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.451878 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.451992 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.452156 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.452291 (+ 135us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.452388 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":121,"threads_started":1}
I20260619 12:24:59.453491 15092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.446271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:34767 (local address 127.0.0.1:37112)
0619 12:24:59.446734 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:59.446785 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:59.446918 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:59.448023 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:59.448029 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:59.448058 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:59.448225 (+ 167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.448233 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.449358 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.449365 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.451057 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.451068 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.453040 (+ 1972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.453045 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.453156 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.453180 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 12:24:59.453259 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 12:24:59.453326 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":69,"thread_start_us":81,"threads_started":1}
I20260619 12:24:59.453584 15093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.446367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37112 (local address 127.6.231.195:34767)
0619 12:24:59.447458 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.447471 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.447495 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:24:59.447578 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.447584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.447839 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.448383 (+ 544us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:59.448520 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.449238 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.452213 (+ 2975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.452925 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.453014 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.453138 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.453311 (+ 173us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.453407 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":976,"thread_start_us":124,"threads_started":1}
I20260619 12:24:59.455467 15095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.447040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:36217 (local address 127.0.0.1:39370)
0619 12:24:59.447399 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:59.447412 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:59.447487 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:59.448352 (+ 865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:59.448354 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:59.448368 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:59.448429 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.448434 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.449743 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.449747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.451057 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.451068 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.455028 (+ 3960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.455032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.455136 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.455165 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 12:24:59.455268 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 12:24:59.455339 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":38,"thread_start_us":97,"threads_started":1}
I20260619 12:24:59.455713 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.447487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39370 (local address 127.6.231.193:36217)
0619 12:24:59.447742 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.447750 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.447773 (+ 23us) server_negotiation.cc:407] Connection header received
0619 12:24:59.447845 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.447850 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.448037 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.448190 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:59.448546 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.449651 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.451184 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.451889 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.451992 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.452148 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.455311 (+ 3163us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.455420 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":79,"threads_started":1}
I20260619 12:24:59.455863 14863 tablet_service.cc:1467] Processing CreateTablet for tablet 7c00d26656164f72855bc2e427e6d3be (DEFAULT_TABLE table=test-table [id=8ae0a6ddff704c16b8feec84f2fe7595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:59.456633 15003 tablet_service.cc:1467] Processing CreateTablet for tablet 7c00d26656164f72855bc2e427e6d3be (DEFAULT_TABLE table=test-table [id=8ae0a6ddff704c16b8feec84f2fe7595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:59.456975 14863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c00d26656164f72855bc2e427e6d3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:59.457631 15003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c00d26656164f72855bc2e427e6d3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:59.459486 14723 tablet_service.cc:1467] Processing CreateTablet for tablet 7c00d26656164f72855bc2e427e6d3be (DEFAULT_TABLE table=test-table [id=8ae0a6ddff704c16b8feec84f2fe7595]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:59.460487 14723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7c00d26656164f72855bc2e427e6d3be. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:59.464160 15097 tablet_bootstrap.cc:492] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b: Bootstrap starting.
I20260619 12:24:59.465773 15098 tablet_bootstrap.cc:492] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25: Bootstrap starting.
I20260619 12:24:59.466354 15097 tablet_bootstrap.cc:654] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:59.467090 15097 log.cc:824] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:59.467803 15098 tablet_bootstrap.cc:654] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:59.468676 15098 log.cc:824] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:59.468673 15100 tablet_bootstrap.cc:492] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6: Bootstrap starting.
I20260619 12:24:59.469125 15097 tablet_bootstrap.cc:492] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b: No bootstrap required, opened a new log
I20260619 12:24:59.469316 15097 ts_tablet_manager.cc:1397] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 12:24:59.470312 15098 tablet_bootstrap.cc:492] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25: No bootstrap required, opened a new log
I20260619 12:24:59.470506 15098 ts_tablet_manager.cc:1397] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 12:24:59.470763 15100 tablet_bootstrap.cc:654] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:59.471553 15100 log.cc:824] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6: Log is configured to *not* fsync() on all Append() calls
I20260619 12:24:59.473619 15100 tablet_bootstrap.cc:492] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6: No bootstrap required, opened a new log
I20260619 12:24:59.473425 15097 raft_consensus.cc:348] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } }
I20260619 12:24:59.473728 15097 raft_consensus.cc:374] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:59.473820 15100 ts_tablet_manager.cc:1397] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260619 12:24:59.473822 15097 raft_consensus.cc:729] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3c56beeca344caa27c3cfe57e57a0b, State: Initialized, Role: FOLLOWER
I20260619 12:24:59.473788 15098 raft_consensus.cc:348] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } }
I20260619 12:24:59.474067 15098 raft_consensus.cc:374] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:59.474152 15098 raft_consensus.cc:729] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657d8ebe44b74eb6952d70a0d44fbb25, State: Initialized, Role: FOLLOWER
I20260619 12:24:59.474290 15097 consensus_queue.cc:260] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } }
I20260619 12:24:59.474561 15098 consensus_queue.cc:260] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } }
I20260619 12:24:59.475414 15097 ts_tablet_manager.cc:1428] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 12:24:59.475472 15098 ts_tablet_manager.cc:1428] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 12:24:59.475795 14929 heartbeater.cc:503] Master 127.6.231.254:36725 was elected leader, sending a full tablet report...
I20260619 12:24:59.475795 15069 heartbeater.cc:503] Master 127.6.231.254:36725 was elected leader, sending a full tablet report...
I20260619 12:24:59.478169 15100 raft_consensus.cc:348] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } }
I20260619 12:24:59.478463 15100 raft_consensus.cc:374] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:59.478546 15100 raft_consensus.cc:729] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1007bb3e7e554133beb50e603f7f7aa6, State: Initialized, Role: FOLLOWER
I20260619 12:24:59.478989 15100 consensus_queue.cc:260] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } }
I20260619 12:24:59.479601 14789 heartbeater.cc:503] Master 127.6.231.254:36725 was elected leader, sending a full tablet report...
I20260619 12:24:59.479971 15100 ts_tablet_manager.cc:1428] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260619 12:24:59.631884 15070 tablet.cc:2369] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:24:59.651312 14930 tablet.cc:2369] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:24:59.656069 15103 raft_consensus.cc:482] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:59.656291 15103 raft_consensus.cc:504] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } }
I20260619 12:24:59.657410 15103 leader_election.cc:288] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657d8ebe44b74eb6952d70a0d44fbb25 (127.6.231.194:35953), 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193:36217)
I20260619 12:24:59.661854 14962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.657709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:35953 (local address 127.6.231.195:60729)
0619 12:24:59.657865 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:59.657882 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:59.657996 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:59.658403 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:59.658406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:59.658420 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:59.658560 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.658566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.659612 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.659615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.660690 (+ 1075us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.660699 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.661371 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.661375 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.661513 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.661536 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 12:24:59.661626 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 12:24:59.661705 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57,"mutex_wait_us":1}
I20260619 12:24:59.661854 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.657783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:60729 (local address 127.6.231.194:35953)
0619 12:24:59.657924 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.657931 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.657948 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:24:59.658146 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.658149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.658201 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.658280 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:59.658685 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.659494 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.660820 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.661236 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.661383 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.661466 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.661640 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.661713 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:24:59.662575 14883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c00d26656164f72855bc2e427e6d3be" candidate_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" is_pre_election: true
I20260619 12:24:59.662937 14883 raft_consensus.cc:2454] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c3c56beeca344caa27c3cfe57e57a0b in term 0.
I20260619 12:24:59.663041 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.658161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:33743 (local address 127.6.231.193:36217)
0619 12:24:59.658293 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.658297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.658479 (+ 182us) server_negotiation.cc:407] Connection header received
0619 12:24:59.658578 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.658581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.658640 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.658710 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:59.659339 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.660016 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.661730 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.662377 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.662464 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.662508 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.662822 (+ 314us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.662910 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:24:59.663419 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.657966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:36217 (local address 127.6.231.195:33743)
0619 12:24:59.658361 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:59.658375 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:59.658496 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:59.658728 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:59.658730 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:59.658744 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:59.659210 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.659217 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.660146 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.660155 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.661591 (+ 1436us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.661601 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.662556 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.662560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.662657 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.662678 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 12:24:59.663172 (+ 494us) client_negotiation.cc:240] Negotiation successful
0619 12:24:59.663264 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":142,"threads_started":1}
I20260619 12:24:59.663502 14956 leader_election.cc:302] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3c56beeca344caa27c3cfe57e57a0b, 657d8ebe44b74eb6952d70a0d44fbb25; no voters:
I20260619 12:24:59.663812 15103 raft_consensus.cc:2781] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:59.663924 15103 raft_consensus.cc:482] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:59.664013 15103 raft_consensus.cc:3037] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:59.664307 14743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c00d26656164f72855bc2e427e6d3be" candidate_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1007bb3e7e554133beb50e603f7f7aa6" is_pre_election: true
I20260619 12:24:59.664567 14743 raft_consensus.cc:2454] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2c3c56beeca344caa27c3cfe57e57a0b in term 0.
I20260619 12:24:59.665300 15103 raft_consensus.cc:504] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } }
I20260619 12:24:59.665673 15103 leader_election.cc:288] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [CANDIDATE]: Term 1 election: Requested vote from peers 657d8ebe44b74eb6952d70a0d44fbb25 (127.6.231.194:35953), 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193:36217)
I20260619 12:24:59.666038 14883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c00d26656164f72855bc2e427e6d3be" candidate_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657d8ebe44b74eb6952d70a0d44fbb25"
I20260619 12:24:59.666044 14743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7c00d26656164f72855bc2e427e6d3be" candidate_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1007bb3e7e554133beb50e603f7f7aa6"
I20260619 12:24:59.666150 14743 raft_consensus.cc:3037] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:59.666173 14883 raft_consensus.cc:3037] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:59.667157 14883 raft_consensus.cc:2454] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c3c56beeca344caa27c3cfe57e57a0b in term 1.
I20260619 12:24:59.667378 14743 raft_consensus.cc:2454] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2c3c56beeca344caa27c3cfe57e57a0b in term 1.
I20260619 12:24:59.667486 14956 leader_election.cc:302] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3c56beeca344caa27c3cfe57e57a0b, 657d8ebe44b74eb6952d70a0d44fbb25; no voters:
I20260619 12:24:59.667654 15103 raft_consensus.cc:2781] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:59.667893 15103 raft_consensus.cc:686] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 LEADER]: Becoming Leader. State: Replica: 2c3c56beeca344caa27c3cfe57e57a0b, State: Running, Role: LEADER
I20260619 12:24:59.668234 15103 consensus_queue.cc:237] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } }
W20260619 12:24:59.671849 14790 tablet.cc:2369] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:24:59.671537 14583 catalog_manager.cc:5495] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b reported cstate change: term changed from 0 to 1, leader changed from <none> to 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195). New cstate: current_term: 1 leader_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } health_report { overall_health: UNKNOWN } } }
I20260619 12:24:59.687800 15093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.683857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37122 (local address 127.6.231.195:34767)
0619 12:24:59.683995 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.683998 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.684015 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:24:59.684203 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.684205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.684255 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.684340 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:24:59.684647 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.685159 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.686267 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.686473 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.686551 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.686902 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:24:59.687334 (+ 432us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:24:59.687429 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.687520 (+ 91us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.687571 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 12:24:59.693818 14743 raft_consensus.cc:1261] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Refusing update from remote peer 2c3c56beeca344caa27c3cfe57e57a0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:59.693811 14883 raft_consensus.cc:1261] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Refusing update from remote peer 2c3c56beeca344caa27c3cfe57e57a0b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:24:59.694514 15107 consensus_queue.cc:1035] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:59.694695 15103 consensus_queue.cc:1035] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [LEADER]: Connected to new peer: Peer: permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:24:59.703065 15112 mvcc.cc:205] Tried to move back new op lower bound from 7298547301139161088 to 7298547301043896320. Current Snapshot: MvccSnapshot[applied={T|T < 7298547301139161088}]
I20260619 12:24:59.703382 15111 mvcc.cc:205] Tried to move back new op lower bound from 7298547301139161088 to 7298547301043896320. Current Snapshot: MvccSnapshot[applied={T|T < 7298547301139161088}]
I20260619 12:24:59.711414 14586 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:59.711611 14586 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:24:59.714076 14586 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6: Sending DeleteTablet for 3 replicas of tablet 7c00d26656164f72855bc2e427e6d3be
I20260619 12:24:59.714823 15003 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c00d26656164f72855bc2e427e6d3be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:59 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:37112
I20260619 12:24:59.714975 14723 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c00d26656164f72855bc2e427e6d3be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:59 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:39370
I20260619 12:24:59.715452 15120 tablet_replica.cc:330] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b: stopping tablet replica
I20260619 12:24:59.715451 14586 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 12:24:59.715616 15121 tablet_replica.cc:330] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6: stopping tablet replica
I20260619 12:24:59.715845 15120 raft_consensus.cc:2229] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 LEADER]: Raft consensus shutting down.
W20260619 12:24:59.715862 14586 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:59.715903 15121 raft_consensus.cc:2229] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:59.716197 15120 raft_consensus.cc:2258] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:59.716202 15121 raft_consensus.cc:2258] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:59.716657 14863 tablet_service.cc:1514] Processing DeleteTablet for tablet 7c00d26656164f72855bc2e427e6d3be with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:24:59 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:39614
I20260619 12:24:59.717630 15120 ts_tablet_manager.cc:1905] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:59.717691 15121 ts_tablet_manager.cc:1905] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:59.717928 15122 tablet_replica.cc:330] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25: stopping tablet replica
I20260619 12:24:59.718238 15122 raft_consensus.cc:2229] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:24:59.718523 15122 raft_consensus.cc:2258] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:24:59.720747 15120 ts_tablet_manager.cc:1918] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:59.720897 15120 log.cc:1196] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/7c00d26656164f72855bc2e427e6d3be
I20260619 12:24:59.721297 15120 ts_tablet_manager.cc:1939] T 7c00d26656164f72855bc2e427e6d3be P 2c3c56beeca344caa27c3cfe57e57a0b: Deleting consensus metadata
I20260619 12:24:59.722524 15121 ts_tablet_manager.cc:1918] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:59.722705 15121 log.cc:1196] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/wal/wals/7c00d26656164f72855bc2e427e6d3be
I20260619 12:24:59.722735 14723 tablet_service.cc:1467] Processing CreateTablet for tablet f52fda54d1dc4c5abf836991db40cfe6 (DEFAULT_TABLE table=test-table [id=68203117d7524427b02351a5a6934ced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:59.723016 14723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f52fda54d1dc4c5abf836991db40cfe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:59.723064 15121 ts_tablet_manager.cc:1939] T 7c00d26656164f72855bc2e427e6d3be P 1007bb3e7e554133beb50e603f7f7aa6: Deleting consensus metadata
I20260619 12:24:59.723062 15003 tablet_service.cc:1467] Processing CreateTablet for tablet f52fda54d1dc4c5abf836991db40cfe6 (DEFAULT_TABLE table=test-table [id=68203117d7524427b02351a5a6934ced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:59.723281 15003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f52fda54d1dc4c5abf836991db40cfe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:59.725303 14863 tablet_service.cc:1467] Processing CreateTablet for tablet f52fda54d1dc4c5abf836991db40cfe6 (DEFAULT_TABLE table=test-table [id=68203117d7524427b02351a5a6934ced]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:24:59.725378 14571 catalog_manager.cc:4841] TS 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195:34767): tablet 7c00d26656164f72855bc2e427e6d3be (table test-table [id=8ae0a6ddff704c16b8feec84f2fe7595]) successfully deleted
I20260619 12:24:59.725502 15097 tablet_bootstrap.cc:492] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b: Bootstrap starting.
I20260619 12:24:59.725582 14863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f52fda54d1dc4c5abf836991db40cfe6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:59.725821 14572 catalog_manager.cc:4841] TS 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193:36217): tablet 7c00d26656164f72855bc2e427e6d3be (table test-table [id=8ae0a6ddff704c16b8feec84f2fe7595]) successfully deleted
I20260619 12:24:59.726414 15097 tablet_bootstrap.cc:654] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:59.727269 15100 tablet_bootstrap.cc:492] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6: Bootstrap starting.
I20260619 12:24:59.727895 15098 tablet_bootstrap.cc:492] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25: Bootstrap starting.
I20260619 12:24:59.728016 15100 tablet_bootstrap.cc:654] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:59.728940 15098 tablet_bootstrap.cc:654] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:59.729220 15100 tablet_bootstrap.cc:492] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6: No bootstrap required, opened a new log
I20260619 12:24:59.729293 15100 ts_tablet_manager.cc:1397] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:59.729363 15097 tablet_bootstrap.cc:492] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b: No bootstrap required, opened a new log
I20260619 12:24:59.729465 15097 ts_tablet_manager.cc:1397] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 12:24:59.730033 15097 raft_consensus.cc:348] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.730181 15097 raft_consensus.cc:374] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:59.730221 15097 raft_consensus.cc:729] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3c56beeca344caa27c3cfe57e57a0b, State: Initialized, Role: FOLLOWER
I20260619 12:24:59.730283 15100 raft_consensus.cc:348] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.730336 15097 consensus_queue.cc:260] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.730386 15100 raft_consensus.cc:374] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:59.730417 15100 raft_consensus.cc:729] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1007bb3e7e554133beb50e603f7f7aa6, State: Initialized, Role: FOLLOWER
I20260619 12:24:59.730523 15100 consensus_queue.cc:260] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.730759 15100 ts_tablet_manager.cc:1428] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:59.730614 15097 ts_tablet_manager.cc:1428] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:59.733309 15122 ts_tablet_manager.cc:1905] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:24:59.734162 15098 tablet_bootstrap.cc:492] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25: No bootstrap required, opened a new log
I20260619 12:24:59.734268 15098 ts_tablet_manager.cc:1397] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.001s
I20260619 12:24:59.734972 15098 raft_consensus.cc:348] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.735101 15098 raft_consensus.cc:374] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:59.735134 15098 raft_consensus.cc:729] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657d8ebe44b74eb6952d70a0d44fbb25, State: Initialized, Role: FOLLOWER
I20260619 12:24:59.735374 15098 consensus_queue.cc:260] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.736022 15122 ts_tablet_manager.cc:1918] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:24:59.736042 15098 ts_tablet_manager.cc:1428] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260619 12:24:59.736178 15122 log.cc:1196] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/7c00d26656164f72855bc2e427e6d3be
I20260619 12:24:59.736586 15122 ts_tablet_manager.cc:1939] T 7c00d26656164f72855bc2e427e6d3be P 657d8ebe44b74eb6952d70a0d44fbb25: Deleting consensus metadata
I20260619 12:24:59.737775 14572 catalog_manager.cc:4841] TS 657d8ebe44b74eb6952d70a0d44fbb25 (127.6.231.194:35953): tablet 7c00d26656164f72855bc2e427e6d3be (table test-table [id=8ae0a6ddff704c16b8feec84f2fe7595]) successfully deleted
I20260619 12:24:59.791244 15104 raft_consensus.cc:482] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:24:59.791404 15104 raft_consensus.cc:504] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.792567 15104 leader_election.cc:288] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193:36217), 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195:34767)
I20260619 12:24:59.797008 15093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.792915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:40219 (local address 127.6.231.195:34767)
0619 12:24:59.793054 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.793059 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.793238 (+ 179us) server_negotiation.cc:407] Connection header received
0619 12:24:59.793399 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.793402 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.793478 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.793559 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:59.793834 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.794438 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.795821 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.796417 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.796559 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.796616 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.796818 (+ 202us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.796884 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:24:59.797021 15127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.792804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:34767 (local address 127.6.231.194:40219)
0619 12:24:59.793165 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:59.793177 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:59.793283 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:59.793577 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:59.793579 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:59.793595 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:59.793715 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.793720 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.794546 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.794552 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.795697 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.795706 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.796558 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.796562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.796675 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.796702 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 12:24:59.796787 (+ 85us) client_negotiation.cc:240] Negotiation successful
0619 12:24:59.796859 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":109,"threads_started":1}
I20260619 12:24:59.797514 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.793130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:44085 (local address 127.6.231.193:36217)
0619 12:24:59.793246 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.793251 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.793266 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:24:59.793692 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.793695 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.793740 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.793804 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:24:59.794104 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.794849 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.796154 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.796730 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.796837 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.796882 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.797247 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.797364 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:24:59.797904 15126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.792774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:36217 (local address 127.6.231.194:44085)
0619 12:24:59.793152 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:59.793165 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:59.793273 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:59.793863 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:59.793866 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:59.793883 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:24:59.793991 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.793996 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.794963 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.794966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.796037 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.796045 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.796981 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.796985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.797082 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.797124 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 12:24:59.797703 (+ 579us) client_negotiation.cc:240] Negotiation successful
0619 12:24:59.797794 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":70,"threads_started":1}
I20260619 12:24:59.797844 15023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f52fda54d1dc4c5abf836991db40cfe6" candidate_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" is_pre_election: true
I20260619 12:24:59.798079 15023 raft_consensus.cc:2454] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657d8ebe44b74eb6952d70a0d44fbb25 in term 0.
I20260619 12:24:59.798521 14743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f52fda54d1dc4c5abf836991db40cfe6" candidate_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1007bb3e7e554133beb50e603f7f7aa6" is_pre_election: true
I20260619 12:24:59.798542 14815 leader_election.cc:302] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3c56beeca344caa27c3cfe57e57a0b, 657d8ebe44b74eb6952d70a0d44fbb25; no voters:
I20260619 12:24:59.798672 14743 raft_consensus.cc:2454] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657d8ebe44b74eb6952d70a0d44fbb25 in term 0.
I20260619 12:24:59.798791 15104 raft_consensus.cc:2781] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:24:59.798846 15104 raft_consensus.cc:482] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:24:59.798884 15104 raft_consensus.cc:3037] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:59.799880 15104 raft_consensus.cc:504] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.800217 15104 leader_election.cc:288] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [CANDIDATE]: Term 1 election: Requested vote from peers 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193:36217), 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195:34767)
I20260619 12:24:59.800487 14743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f52fda54d1dc4c5abf836991db40cfe6" candidate_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1007bb3e7e554133beb50e603f7f7aa6"
I20260619 12:24:59.800621 15023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f52fda54d1dc4c5abf836991db40cfe6" candidate_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3c56beeca344caa27c3cfe57e57a0b"
I20260619 12:24:59.800678 14743 raft_consensus.cc:3037] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:59.800738 15023 raft_consensus.cc:3037] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:24:59.801833 14743 raft_consensus.cc:2454] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657d8ebe44b74eb6952d70a0d44fbb25 in term 1.
I20260619 12:24:59.801833 15023 raft_consensus.cc:2454] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657d8ebe44b74eb6952d70a0d44fbb25 in term 1.
I20260619 12:24:59.802116 14816 leader_election.cc:302] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1007bb3e7e554133beb50e603f7f7aa6, 657d8ebe44b74eb6952d70a0d44fbb25; no voters:
I20260619 12:24:59.802309 15104 raft_consensus.cc:2781] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:24:59.802541 15104 raft_consensus.cc:686] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 LEADER]: Becoming Leader. State: Replica: 657d8ebe44b74eb6952d70a0d44fbb25, State: Running, Role: LEADER
I20260619 12:24:59.802800 15104 consensus_queue.cc:237] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.805207 14586 catalog_manager.cc:5495] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657d8ebe44b74eb6952d70a0d44fbb25 (127.6.231.194). New cstate: current_term: 1 leader_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } health_report { overall_health: UNKNOWN } } }
Jun 19 12:24:59 dist-test-slave-kr9h krb5kdc[14541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871898, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.254@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
I20260619 12:24:59.856544 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.847197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36725 (local address 127.0.0.1:46738)
0619 12:24:59.847521 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0619 12:24:59.847533 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:24:59.847715 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:24:59.847968 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:24:59.847971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:24:59.848567 (+ 596us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:24:59.848709 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.848720 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.849709 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.849715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.850275 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:24:59.850282 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.850741 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.850744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:24:59.850823 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.851448 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:24:59.851460 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:24:59.852840 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:24:59.854832 (+ 1992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:59.854840 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:59.854847 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:59.855166 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:59.855438 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:24:59.855442 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:24:59.855444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:24:59.855526 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:24:59.855899 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:24:59.855908 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:24:59.856021 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 12:24:59.856146 (+ 125us) client_negotiation.cc:240] Negotiation successful
0619 12:24:59.856314 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":130,"threads_started":1}
I20260619 12:24:59.856549 14635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:24:59.847301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46738 (local address 127.6.231.254:36725)
0619 12:24:59.847444 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 12:24:59.847449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:24:59.847655 (+ 206us) server_negotiation.cc:407] Connection header received
0619 12:24:59.847821 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:24:59.847824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:24:59.847867 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:24:59.847951 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:24:59.848853 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.849605 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.850396 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:24:59.850629 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:24:59.850738 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:24:59.852972 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:24:59.852976 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:24:59.852978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:24:59.853000 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:24:59.854674 (+ 1674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:59.855262 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:59.855268 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:59.855270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:59.855347 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:24:59.855612 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:24:59.855618 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:24:59.855620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:24:59.855787 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:24:59.855890 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 12:24:59.856199 (+ 309us) server_negotiation.cc:299] Negotiation successful
0619 12:24:59.856369 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":59}
I20260619 12:24:59.861251 14586 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:46738:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260619 12:24:59.862221 14586 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:24:59.867108 14723 tablet_service.cc:1467] Processing CreateTablet for tablet 62f87ea7fa7f4a34b983b1aa2db71f28 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83ffc782eb774612b5bd04b3086c00e0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:24:59.867341 15003 tablet_service.cc:1467] Processing CreateTablet for tablet 62f87ea7fa7f4a34b983b1aa2db71f28 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83ffc782eb774612b5bd04b3086c00e0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:24:59.867506 14723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62f87ea7fa7f4a34b983b1aa2db71f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:59.867720 15003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62f87ea7fa7f4a34b983b1aa2db71f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:59.869858 14863 tablet_service.cc:1467] Processing CreateTablet for tablet 62f87ea7fa7f4a34b983b1aa2db71f28 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=83ffc782eb774612b5bd04b3086c00e0]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260619 12:24:59.870047 15097 tablet_bootstrap.cc:492] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b: Bootstrap starting.
I20260619 12:24:59.870222 14863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 62f87ea7fa7f4a34b983b1aa2db71f28. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:24:59.870323 15100 tablet_bootstrap.cc:492] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6: Bootstrap starting.
I20260619 12:24:59.870936 15097 tablet_bootstrap.cc:654] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:59.871243 15100 tablet_bootstrap.cc:654] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:59.872197 15097 tablet_bootstrap.cc:492] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b: No bootstrap required, opened a new log
I20260619 12:24:59.872272 15097 ts_tablet_manager.cc:1397] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260619 12:24:59.872740 15100 tablet_bootstrap.cc:492] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6: No bootstrap required, opened a new log
I20260619 12:24:59.872676 15097 raft_consensus.cc:348] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.872781 15097 raft_consensus.cc:374] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:59.872807 15100 ts_tablet_manager.cc:1397] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:24:59.872813 15097 raft_consensus.cc:729] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3c56beeca344caa27c3cfe57e57a0b, State: Initialized, Role: FOLLOWER
I20260619 12:24:59.872951 15097 consensus_queue.cc:260] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.873225 15097 ts_tablet_manager.cc:1428] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:59.873180 15115 tablet_replica.cc:439] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } }
I20260619 12:24:59.873196 15100 raft_consensus.cc:348] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.873294 15115 tablet_replica.cc:442] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:59.873303 15100 raft_consensus.cc:374] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:59.873330 15100 raft_consensus.cc:729] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1007bb3e7e554133beb50e603f7f7aa6, State: Initialized, Role: FOLLOWER
I20260619 12:24:59.873456 15100 consensus_queue.cc:260] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.873687 15100 ts_tablet_manager.cc:1428] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:24:59.873849 15105 tablet_replica.cc:439] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } }
I20260619 12:24:59.873962 15105 tablet_replica.cc:442] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:24:59.875428 15098 tablet_bootstrap.cc:492] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25: Bootstrap starting.
I20260619 12:24:59.876384 15098 tablet_bootstrap.cc:654] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25: Neither blocks nor log segments found. Creating new log.
I20260619 12:24:59.877615 15098 tablet_bootstrap.cc:492] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25: No bootstrap required, opened a new log
I20260619 12:24:59.877691 15098 ts_tablet_manager.cc:1397] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260619 12:24:59.878005 15098 raft_consensus.cc:348] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.878094 15098 raft_consensus.cc:374] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:24:59.878115 15098 raft_consensus.cc:729] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657d8ebe44b74eb6952d70a0d44fbb25, State: Initialized, Role: FOLLOWER
I20260619 12:24:59.878223 15098 consensus_queue.cc:260] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:24:59.878470 15098 ts_tablet_manager.cc:1428] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260619 12:24:59.878484 15104 tablet_replica.cc:439] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } }
I20260619 12:24:59.878686 15104 tablet_replica.cc:442] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:25:00.114102 15105 raft_consensus.cc:482] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:25:00.114308 15105 raft_consensus.cc:504] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.115469 15105 leader_election.cc:288] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 657d8ebe44b74eb6952d70a0d44fbb25 (127.6.231.194:35953), 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195:34767)
I20260619 12:25:00.121183 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.115933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:36333 (local address 127.6.231.194:35953)
0619 12:25:00.116094 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:00.116100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:00.116343 (+ 243us) server_negotiation.cc:407] Connection header received
0619 12:25:00.116505 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:00.116509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:00.116561 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:00.116652 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:25:00.117330 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.118193 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.120155 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.120635 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.120806 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.120854 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:00.121008 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 12:25:00.121068 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 12:25:00.121279 15140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.115825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:35953 (local address 127.6.231.193:36333)
0619 12:25:00.116222 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:00.116235 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:00.116386 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:00.116738 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:00.116740 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:00.116757 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:25:00.117116 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:00.117124 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.118398 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.118403 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:00.120017 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:00.120026 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.120783 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.120787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:00.120880 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.120911 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 12:25:00.120994 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 12:25:00.121093 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":445,"spinlock_wait_cycles":17280,"thread_start_us":80,"threads_started":1}
I20260619 12:25:00.121343 15093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.115951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:33731 (local address 127.6.231.195:34767)
0619 12:25:00.116247 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:00.116251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:00.116336 (+ 85us) server_negotiation.cc:407] Connection header received
0619 12:25:00.116505 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:00.116509 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:00.116561 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:00.116652 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:25:00.117273 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.118170 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.120183 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.120669 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.120819 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.120871 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:00.121142 (+ 271us) server_negotiation.cc:299] Negotiation successful
0619 12:25:00.121218 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260619 12:25:00.121855 15139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.115825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:34767 (local address 127.6.231.193:33731)
0619 12:25:00.116213 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:00.116231 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:00.116386 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:00.116723 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:00.116728 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:00.116757 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:25:00.117117 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:00.117124 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.118338 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.118343 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:00.120014 (+ 1671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:00.120026 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.120920 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.120923 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:00.121016 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.121035 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 12:25:00.121575 (+ 540us) client_negotiation.cc:240] Negotiation successful
0619 12:25:00.121678 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":257,"spinlock_wait_cycles":22528,"thread_start_us":111,"threads_started":1}
I20260619 12:25:00.121930 14883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62f87ea7fa7f4a34b983b1aa2db71f28" candidate_uuid: "1007bb3e7e554133beb50e603f7f7aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" is_pre_election: true
I20260619 12:25:00.122108 14883 raft_consensus.cc:2454] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1007bb3e7e554133beb50e603f7f7aa6 in term 0.
I20260619 12:25:00.122689 14676 leader_election.cc:302] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1007bb3e7e554133beb50e603f7f7aa6, 657d8ebe44b74eb6952d70a0d44fbb25; no voters:
I20260619 12:25:00.122792 15023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62f87ea7fa7f4a34b983b1aa2db71f28" candidate_uuid: "1007bb3e7e554133beb50e603f7f7aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" is_pre_election: true
I20260619 12:25:00.122934 15023 raft_consensus.cc:2454] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1007bb3e7e554133beb50e603f7f7aa6 in term 0.
I20260619 12:25:00.123040 15105 raft_consensus.cc:2781] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:25:00.123165 15105 raft_consensus.cc:482] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:25:00.123238 15105 raft_consensus.cc:3037] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:00.124784 15105 raft_consensus.cc:504] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.125278 15105 leader_election.cc:288] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [CANDIDATE]: Term 1 election: Requested vote from peers 657d8ebe44b74eb6952d70a0d44fbb25 (127.6.231.194:35953), 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195:34767)
I20260619 12:25:00.125759 15023 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62f87ea7fa7f4a34b983b1aa2db71f28" candidate_uuid: "1007bb3e7e554133beb50e603f7f7aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3c56beeca344caa27c3cfe57e57a0b"
I20260619 12:25:00.125934 15023 raft_consensus.cc:3037] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:00.126266 14883 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "62f87ea7fa7f4a34b983b1aa2db71f28" candidate_uuid: "1007bb3e7e554133beb50e603f7f7aa6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "657d8ebe44b74eb6952d70a0d44fbb25"
I20260619 12:25:00.126416 14883 raft_consensus.cc:3037] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:00.127388 15023 raft_consensus.cc:2454] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1007bb3e7e554133beb50e603f7f7aa6 in term 1.
I20260619 12:25:00.127611 14883 raft_consensus.cc:2454] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1007bb3e7e554133beb50e603f7f7aa6 in term 1.
I20260619 12:25:00.127818 14675 leader_election.cc:302] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1007bb3e7e554133beb50e603f7f7aa6, 2c3c56beeca344caa27c3cfe57e57a0b; no voters:
I20260619 12:25:00.128017 15105 raft_consensus.cc:2781] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:25:00.128237 15105 raft_consensus.cc:686] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 LEADER]: Becoming Leader. State: Replica: 1007bb3e7e554133beb50e603f7f7aa6, State: Running, Role: LEADER
I20260619 12:25:00.128510 15105 consensus_queue.cc:237] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.130996 15141 tablet_replica.cc:439] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6: TxnStatusTablet state changed. Reason: New leader 1007bb3e7e554133beb50e603f7f7aa6. Latest consensus state: current_term: 1 leader_uuid: "1007bb3e7e554133beb50e603f7f7aa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } }
I20260619 12:25:00.131101 15141 tablet_replica.cc:442] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:25:00.131376 15143 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260619 12:25:00.131527 15143 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260619 12:25:00.131639 14586 catalog_manager.cc:5495] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193). New cstate: current_term: 1 leader_uuid: "1007bb3e7e554133beb50e603f7f7aa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } health_report { overall_health: UNKNOWN } } }
I20260619 12:25:00.173662 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.169685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:36217 (local address 127.0.0.1:39384)
0619 12:25:00.169820 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:00.169837 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:00.169947 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:00.170342 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:00.170345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:00.170352 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:25:00.170461 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:00.170475 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.171218 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.171222 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:00.171862 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:00.171870 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.172340 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.172344 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:00.172436 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.172496 (+ 60us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:25:00.173318 (+ 822us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:25:00.173334 (+ 16us) client_negotiation.cc:769] Sending connection context
0619 12:25:00.173410 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 12:25:00.173531 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260619 12:25:00.173667 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.169829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39384 (local address 127.6.231.193:36217)
0619 12:25:00.169950 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:00.169955 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:00.169969 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:25:00.170150 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:00.170153 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:00.170223 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:00.170330 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:25:00.170579 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.171103 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.171991 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.172228 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.172340 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.172655 (+ 315us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:25:00.173213 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:25:00.173341 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:00.173421 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 12:25:00.173508 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260619 12:25:00.176728 14883 raft_consensus.cc:1261] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Refusing update from remote peer 1007bb3e7e554133beb50e603f7f7aa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:25:00.176923 15023 raft_consensus.cc:1261] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Refusing update from remote peer 1007bb3e7e554133beb50e603f7f7aa6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:25:00.177397 15141 consensus_queue.cc:1035] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:25:00.177681 15105 consensus_queue.cc:1035] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:25:00.179458 15115 tablet_replica.cc:439] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b: TxnStatusTablet state changed. Reason: New leader 1007bb3e7e554133beb50e603f7f7aa6. Latest consensus state: current_term: 1 leader_uuid: "1007bb3e7e554133beb50e603f7f7aa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } }
I20260619 12:25:00.179597 15115 tablet_replica.cc:442] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:25:00.179558 15104 tablet_replica.cc:439] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25: TxnStatusTablet state changed. Reason: New leader 1007bb3e7e554133beb50e603f7f7aa6. Latest consensus state: current_term: 1 leader_uuid: "1007bb3e7e554133beb50e603f7f7aa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } }
I20260619 12:25:00.179689 15104 tablet_replica.cc:442] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:25:00.181860 15105 tablet_replica.cc:439] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1007bb3e7e554133beb50e603f7f7aa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } }
I20260619 12:25:00.181968 15105 tablet_replica.cc:442] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:25:00.183224 15110 mvcc.cc:205] Tried to move back new op lower bound from 7298547303118544896 to 7298547302929612800. Current Snapshot: MvccSnapshot[applied={T|T < 7298547303118544896}]
I20260619 12:25:00.183388 15115 tablet_replica.cc:439] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1007bb3e7e554133beb50e603f7f7aa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } }
I20260619 12:25:00.183499 15115 tablet_replica.cc:442] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 2c3c56beeca344caa27c3cfe57e57a0b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:25:00.184289 15105 tablet_replica.cc:439] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1007bb3e7e554133beb50e603f7f7aa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } }
I20260619 12:25:00.184405 15105 tablet_replica.cc:442] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 1007bb3e7e554133beb50e603f7f7aa6: This TxnStatusTablet replica's current role is: LEADER
I20260619 12:25:00.185565 15104 tablet_replica.cc:439] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1007bb3e7e554133beb50e603f7f7aa6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } } }
I20260619 12:25:00.185688 15104 tablet_replica.cc:442] T 62f87ea7fa7f4a34b983b1aa2db71f28 P 657d8ebe44b74eb6952d70a0d44fbb25: This TxnStatusTablet replica's current role is: FOLLOWER
I20260619 12:25:00.192663 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.188224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39616 (local address 127.6.231.194:35953)
0619 12:25:00.188364 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:00.188369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:00.188658 (+ 289us) server_negotiation.cc:407] Connection header received
0619 12:25:00.188733 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:00.188735 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:00.188797 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:00.188885 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:25:00.189210 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.189900 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.191181 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.191412 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.191691 (+ 279us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.191796 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:25:00.192249 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:25:00.192343 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:00.192439 (+ 96us) server_negotiation.cc:299] Negotiation successful
0619 12:25:00.192489 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:25:00.204551 15096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.200941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:40477 (local address 127.6.231.193:36217)
0619 12:25:00.201075 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:00.201080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:00.201214 (+ 134us) server_negotiation.cc:407] Connection header received
0619 12:25:00.201383 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:00.201386 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:00.201483 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:00.201556 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:25:00.201809 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.202350 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.203376 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.203613 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.203733 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.204031 (+ 298us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:25:00.204121 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:25:00.204214 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:00.204332 (+ 118us) server_negotiation.cc:299] Negotiation successful
0619 12:25:00.204386 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 12:25:00.204564 15152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.200794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:36217 (local address 127.6.231.194:40477)
0619 12:25:00.201105 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:00.201133 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:00.201276 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:00.201568 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:00.201571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:00.201578 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:25:00.201702 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:00.201707 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.202468 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.202471 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:00.203236 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:00.203246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.203736 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.203740 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:00.203837 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.203884 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:25:00.204220 (+ 336us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:25:00.204229 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 12:25:00.204319 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 12:25:00.204401 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":96,"threads_started":1}
I20260619 12:25:00.213413 14743 raft_consensus.cc:1261] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Refusing update from remote peer 657d8ebe44b74eb6952d70a0d44fbb25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:25:00.213514 15023 raft_consensus.cc:1261] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Refusing update from remote peer 657d8ebe44b74eb6952d70a0d44fbb25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:25:00.213984 15104 consensus_queue.cc:1035] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:25:00.214181 15128 consensus_queue.cc:1035] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:25:00.237880 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.233901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:50117 (local address 127.6.231.194:35953)
0619 12:25:00.234025 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:00.234030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:00.234044 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:25:00.234208 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:00.234211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:00.234273 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:00.234352 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:25:00.234594 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.235411 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.236738 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.236949 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.237030 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.237361 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:25:00.237484 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:25:00.237555 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:00.237657 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 12:25:00.237722 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260619 12:25:00.238297 15158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.233670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:35953 (local address 127.6.231.193:50117)
0619 12:25:00.233950 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:00.233966 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:00.234087 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:00.234364 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:00.234368 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:00.234376 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0619 12:25:00.234487 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:00.234498 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.235629 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.235633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:00.236616 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:00.236628 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.237057 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.237060 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:00.237159 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.237225 (+ 66us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0619 12:25:00.237556 (+ 331us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0619 12:25:00.237563 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:25:00.237642 (+ 79us) client_negotiation.cc:240] Negotiation successful
0619 12:25:00.237920 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":78,"threads_started":1}
I20260619 12:25:00.267328 14586 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:25:00.267532 14586 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:25:00.270102 14586 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6: Sending DeleteTablet for 3 replicas of tablet f52fda54d1dc4c5abf836991db40cfe6
I20260619 12:25:00.270916 15003 tablet_service.cc:1514] Processing DeleteTablet for tablet f52fda54d1dc4c5abf836991db40cfe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:25:00 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:37112
I20260619 12:25:00.271379 14723 tablet_service.cc:1514] Processing DeleteTablet for tablet f52fda54d1dc4c5abf836991db40cfe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:25:00 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:39370
I20260619 12:25:00.271775 15162 tablet_replica.cc:330] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6: stopping tablet replica
I20260619 12:25:00.271921 15162 raft_consensus.cc:2229] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:25:00.272048 15162 raft_consensus.cc:2258] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:25:00.272188 14863 tablet_service.cc:1514] Processing DeleteTablet for tablet f52fda54d1dc4c5abf836991db40cfe6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:25:00 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:39614
I20260619 12:25:00.272521 15163 tablet_replica.cc:330] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25: stopping tablet replica
I20260619 12:25:00.272637 15163 raft_consensus.cc:2229] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:25:00.272962 15163 raft_consensus.cc:2258] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:25:00.273739 15163 ts_tablet_manager.cc:1905] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:25:00.276909 15163 ts_tablet_manager.cc:1918] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:25:00.277000 15163 log.cc:1196] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/f52fda54d1dc4c5abf836991db40cfe6
I20260619 12:25:00.277135 15162 ts_tablet_manager.cc:1905] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:25:00.277294 15163 ts_tablet_manager.cc:1939] T f52fda54d1dc4c5abf836991db40cfe6 P 657d8ebe44b74eb6952d70a0d44fbb25: Deleting consensus metadata
I20260619 12:25:00.278146 14572 catalog_manager.cc:4841] TS 657d8ebe44b74eb6952d70a0d44fbb25 (127.6.231.194:35953): tablet f52fda54d1dc4c5abf836991db40cfe6 (table test-table [id=68203117d7524427b02351a5a6934ced]) successfully deleted
I20260619 12:25:00.280892 15162 ts_tablet_manager.cc:1918] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:25:00.280992 15162 log.cc:1196] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/wal/wals/f52fda54d1dc4c5abf836991db40cfe6
I20260619 12:25:00.281291 15162 ts_tablet_manager.cc:1939] T f52fda54d1dc4c5abf836991db40cfe6 P 1007bb3e7e554133beb50e603f7f7aa6: Deleting consensus metadata
I20260619 12:25:00.282151 14572 catalog_manager.cc:4841] TS 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193:36217): tablet f52fda54d1dc4c5abf836991db40cfe6 (table test-table [id=68203117d7524427b02351a5a6934ced]) successfully deleted
I20260619 12:25:00.282519 15161 tablet_replica.cc:330] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b: stopping tablet replica
I20260619 12:25:00.282671 15161 raft_consensus.cc:2229] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:25:00.282814 15161 raft_consensus.cc:2258] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:25:00.283595 15161 ts_tablet_manager.cc:1905] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:25:00.287110 15161 ts_tablet_manager.cc:1918] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:25:00.287210 15161 log.cc:1196] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/f52fda54d1dc4c5abf836991db40cfe6
I20260619 12:25:00.287493 15161 ts_tablet_manager.cc:1939] T f52fda54d1dc4c5abf836991db40cfe6 P 2c3c56beeca344caa27c3cfe57e57a0b: Deleting consensus metadata
I20260619 12:25:00.288359 14571 catalog_manager.cc:4841] TS 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195:34767): tablet f52fda54d1dc4c5abf836991db40cfe6 (table test-table [id=68203117d7524427b02351a5a6934ced]) successfully deleted
Jun 19 12:25:00 dist-test-slave-kr9h krb5kdc[14541](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Jun 19 12:25:00 dist-test-slave-kr9h krb5kdc[14541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:00 dist-test-slave-kr9h krb5kdc[14541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 12:25:00.313832 14635 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:00.305286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46746 (local address 127.6.231.254:36725)
0619 12:25:00.305469 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:00.305474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:00.305779 (+ 305us) server_negotiation.cc:407] Connection header received
0619 12:25:00.305911 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:00.305915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:00.305973 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:00.306074 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:00.306996 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.307808 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.309088 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.309492 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.309691 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.313668 (+ 3977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:46746: BlockingRecv error: recv got EOF from 127.0.0.1:46746 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 12:25:00 dist-test-slave-kr9h krb5kdc[14541](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM
I20260619 12:25:00.326807 14635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.316046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46762 (local address 127.6.231.254:36725)
0619 12:25:00.316168 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:00.316172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:00.316367 (+ 195us) server_negotiation.cc:407] Connection header received
0619 12:25:00.316476 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:00.316479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:00.316528 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:00.316619 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:00.317379 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.318256 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.319465 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.319802 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.319912 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.323031 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:00.323037 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:00.323041 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:25:00.323092 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:00.324919 (+ 1827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:00.325415 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:00.325419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:00.325421 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:00.325533 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:00.325807 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:00.325810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:00.325811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:00.326035 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:00.326173 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:00.326518 (+ 345us) server_negotiation.cc:299] Negotiation successful
0619 12:25:00.326650 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 12:25:00.330454 14586 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:25:00.330956 14586 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:25:00.336057 14723 tablet_service.cc:1467] Processing CreateTablet for tablet b90953423cd04fc5962e5b11c6864439 (DEFAULT_TABLE table=test-table [id=31270f709d4d4e5b80951c307f0cadf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:25:00.336215 15003 tablet_service.cc:1467] Processing CreateTablet for tablet b90953423cd04fc5962e5b11c6864439 (DEFAULT_TABLE table=test-table [id=31270f709d4d4e5b80951c307f0cadf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:25:00.336452 14723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b90953423cd04fc5962e5b11c6864439. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:25:00.336472 15003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b90953423cd04fc5962e5b11c6864439. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:25:00.338989 15100 tablet_bootstrap.cc:492] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6: Bootstrap starting.
I20260619 12:25:00.339233 14863 tablet_service.cc:1467] Processing CreateTablet for tablet b90953423cd04fc5962e5b11c6864439 (DEFAULT_TABLE table=test-table [id=31270f709d4d4e5b80951c307f0cadf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:25:00.339533 14863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b90953423cd04fc5962e5b11c6864439. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:25:00.339892 15100 tablet_bootstrap.cc:654] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6: Neither blocks nor log segments found. Creating new log.
I20260619 12:25:00.340521 15097 tablet_bootstrap.cc:492] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b: Bootstrap starting.
I20260619 12:25:00.341392 15097 tablet_bootstrap.cc:654] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b: Neither blocks nor log segments found. Creating new log.
I20260619 12:25:00.341675 15098 tablet_bootstrap.cc:492] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25: Bootstrap starting.
I20260619 12:25:00.342548 15098 tablet_bootstrap.cc:654] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25: Neither blocks nor log segments found. Creating new log.
I20260619 12:25:00.342631 15100 tablet_bootstrap.cc:492] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6: No bootstrap required, opened a new log
I20260619 12:25:00.342715 15100 ts_tablet_manager.cc:1397] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260619 12:25:00.343160 15100 raft_consensus.cc:348] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.343291 15100 raft_consensus.cc:374] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:25:00.343323 15100 raft_consensus.cc:729] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1007bb3e7e554133beb50e603f7f7aa6, State: Initialized, Role: FOLLOWER
I20260619 12:25:00.343468 15100 consensus_queue.cc:260] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.343788 15100 ts_tablet_manager.cc:1428] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:25:00.344117 15098 tablet_bootstrap.cc:492] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25: No bootstrap required, opened a new log
I20260619 12:25:00.344199 15098 ts_tablet_manager.cc:1397] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 12:25:00.344630 15098 raft_consensus.cc:348] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.344756 15098 raft_consensus.cc:374] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:25:00.344794 15098 raft_consensus.cc:729] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657d8ebe44b74eb6952d70a0d44fbb25, State: Initialized, Role: FOLLOWER
I20260619 12:25:00.344815 15097 tablet_bootstrap.cc:492] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b: No bootstrap required, opened a new log
I20260619 12:25:00.344916 15097 ts_tablet_manager.cc:1397] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260619 12:25:00.344934 15098 consensus_queue.cc:260] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.345191 15098 ts_tablet_manager.cc:1428] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:25:00.345305 15097 raft_consensus.cc:348] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.345419 15097 raft_consensus.cc:374] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:25:00.346402 15097 raft_consensus.cc:729] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3c56beeca344caa27c3cfe57e57a0b, State: Initialized, Role: FOLLOWER
I20260619 12:25:00.346527 15097 consensus_queue.cc:260] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.346817 15097 ts_tablet_manager.cc:1428] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260619 12:25:00.379840 15156 raft_consensus.cc:482] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:25:00.379990 15156 raft_consensus.cc:504] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.380530 15156 leader_election.cc:288] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193:36217), 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195:34767)
I20260619 12:25:00.380952 14743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b90953423cd04fc5962e5b11c6864439" candidate_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1007bb3e7e554133beb50e603f7f7aa6" is_pre_election: true
I20260619 12:25:00.381102 14743 raft_consensus.cc:2454] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657d8ebe44b74eb6952d70a0d44fbb25 in term 0.
I20260619 12:25:00.381126 15022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b90953423cd04fc5962e5b11c6864439" candidate_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" is_pre_election: true
I20260619 12:25:00.381273 15022 raft_consensus.cc:2454] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657d8ebe44b74eb6952d70a0d44fbb25 in term 0.
I20260619 12:25:00.381464 14816 leader_election.cc:302] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1007bb3e7e554133beb50e603f7f7aa6, 657d8ebe44b74eb6952d70a0d44fbb25; no voters:
I20260619 12:25:00.381636 15156 raft_consensus.cc:2781] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:25:00.381680 15156 raft_consensus.cc:482] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:25:00.381716 15156 raft_consensus.cc:3037] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:00.383018 15156 raft_consensus.cc:504] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.383389 15156 leader_election.cc:288] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [CANDIDATE]: Term 1 election: Requested vote from peers 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193:36217), 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195:34767)
I20260619 12:25:00.383811 14743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b90953423cd04fc5962e5b11c6864439" candidate_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1007bb3e7e554133beb50e603f7f7aa6"
I20260619 12:25:00.383944 14743 raft_consensus.cc:3037] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:00.384497 15022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b90953423cd04fc5962e5b11c6864439" candidate_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3c56beeca344caa27c3cfe57e57a0b"
I20260619 12:25:00.384629 15022 raft_consensus.cc:3037] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:00.385455 14743 raft_consensus.cc:2454] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657d8ebe44b74eb6952d70a0d44fbb25 in term 1.
I20260619 12:25:00.385782 15022 raft_consensus.cc:2454] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657d8ebe44b74eb6952d70a0d44fbb25 in term 1.
I20260619 12:25:00.385782 14816 leader_election.cc:302] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1007bb3e7e554133beb50e603f7f7aa6, 657d8ebe44b74eb6952d70a0d44fbb25; no voters:
I20260619 12:25:00.385946 15156 raft_consensus.cc:2781] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:25:00.386014 15156 raft_consensus.cc:686] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 LEADER]: Becoming Leader. State: Replica: 657d8ebe44b74eb6952d70a0d44fbb25, State: Running, Role: LEADER
I20260619 12:25:00.386179 15156 consensus_queue.cc:237] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.387959 14586 catalog_manager.cc:5495] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657d8ebe44b74eb6952d70a0d44fbb25 (127.6.231.194). New cstate: current_term: 1 leader_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } health_report { overall_health: UNKNOWN } } }
I20260619 12:25:00.403218 15094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.398814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39624 (local address 127.6.231.194:35953)
0619 12:25:00.398940 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:00.398944 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:00.398959 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:25:00.399058 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:00.399060 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:00.399109 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:00.399186 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:25:00.399558 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.400346 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.401693 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:00.402050 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:00.402140 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:00.402474 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:25:00.402561 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:25:00.402616 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:00.403004 (+ 388us) server_negotiation.cc:299] Negotiation successful
0619 12:25:00.403075 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:25:00.405385 14743 raft_consensus.cc:1261] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Refusing update from remote peer 657d8ebe44b74eb6952d70a0d44fbb25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:25:00.405795 15022 raft_consensus.cc:1261] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Refusing update from remote peer 657d8ebe44b74eb6952d70a0d44fbb25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:25:00.405937 15156 consensus_queue.cc:1035] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:25:00.406229 15128 consensus_queue.cc:1035] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:25:00.416951 14586 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:25:00.417142 14586 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:25:00.419366 14586 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6: Sending DeleteTablet for 3 replicas of tablet b90953423cd04fc5962e5b11c6864439
I20260619 12:25:00.420060 14723 tablet_service.cc:1514] Processing DeleteTablet for tablet b90953423cd04fc5962e5b11c6864439 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:25:00 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:39370
I20260619 12:25:00.420140 15003 tablet_service.cc:1514] Processing DeleteTablet for tablet b90953423cd04fc5962e5b11c6864439 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:25:00 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:37112
I20260619 12:25:00.420284 15162 tablet_replica.cc:330] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6: stopping tablet replica
I20260619 12:25:00.420313 14863 tablet_service.cc:1514] Processing DeleteTablet for tablet b90953423cd04fc5962e5b11c6864439 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:25:00 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:39614
I20260619 12:25:00.420406 15162 raft_consensus.cc:2229] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:25:00.420480 15163 tablet_replica.cc:330] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25: stopping tablet replica
I20260619 12:25:00.420511 15161 tablet_replica.cc:330] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b: stopping tablet replica
I20260619 12:25:00.420571 15162 raft_consensus.cc:2258] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:25:00.420619 15161 raft_consensus.cc:2229] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:25:00.420619 15163 raft_consensus.cc:2229] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:25:00.420735 15161 raft_consensus.cc:2258] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:25:00.420800 15163 raft_consensus.cc:2258] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:25:00.421411 15162 ts_tablet_manager.cc:1905] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:25:00.421690 15161 ts_tablet_manager.cc:1905] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:25:00.424032 15162 ts_tablet_manager.cc:1918] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:25:00.424041 15161 ts_tablet_manager.cc:1918] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:25:00.424121 15161 log.cc:1196] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/b90953423cd04fc5962e5b11c6864439
I20260619 12:25:00.424125 15162 log.cc:1196] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-0/wal/wals/b90953423cd04fc5962e5b11c6864439
I20260619 12:25:00.424386 15162 ts_tablet_manager.cc:1939] T b90953423cd04fc5962e5b11c6864439 P 1007bb3e7e554133beb50e603f7f7aa6: Deleting consensus metadata
I20260619 12:25:00.424401 15161 ts_tablet_manager.cc:1939] T b90953423cd04fc5962e5b11c6864439 P 2c3c56beeca344caa27c3cfe57e57a0b: Deleting consensus metadata
I20260619 12:25:00.425347 14571 catalog_manager.cc:4841] TS 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195:34767): tablet b90953423cd04fc5962e5b11c6864439 (table test-table [id=31270f709d4d4e5b80951c307f0cadf6]) successfully deleted
I20260619 12:25:00.425475 15163 ts_tablet_manager.cc:1905] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:25:00.425884 14572 catalog_manager.cc:4841] TS 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193:36217): tablet b90953423cd04fc5962e5b11c6864439 (table test-table [id=31270f709d4d4e5b80951c307f0cadf6]) successfully deleted
I20260619 12:25:00.426066 14586 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46762:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:25:00.426528 14586 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:25:00.427584 15163 ts_tablet_manager.cc:1918] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:25:00.427676 15163 log.cc:1196] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/b90953423cd04fc5962e5b11c6864439
I20260619 12:25:00.427944 15163 ts_tablet_manager.cc:1939] T b90953423cd04fc5962e5b11c6864439 P 657d8ebe44b74eb6952d70a0d44fbb25: Deleting consensus metadata
I20260619 12:25:00.428675 14572 catalog_manager.cc:4841] TS 657d8ebe44b74eb6952d70a0d44fbb25 (127.6.231.194:35953): tablet b90953423cd04fc5962e5b11c6864439 (table test-table [id=31270f709d4d4e5b80951c307f0cadf6]) successfully deleted
I20260619 12:25:00.432137 14863 tablet_service.cc:1467] Processing CreateTablet for tablet 746f3456c10949c3b890020b1bd6e143 (DEFAULT_TABLE table=test-table [id=63ebb53627584b9eb28e9578110b748b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:25:00.432137 14723 tablet_service.cc:1467] Processing CreateTablet for tablet 746f3456c10949c3b890020b1bd6e143 (DEFAULT_TABLE table=test-table [id=63ebb53627584b9eb28e9578110b748b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:25:00.432297 15003 tablet_service.cc:1467] Processing CreateTablet for tablet 746f3456c10949c3b890020b1bd6e143 (DEFAULT_TABLE table=test-table [id=63ebb53627584b9eb28e9578110b748b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:25:00.432446 14723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 746f3456c10949c3b890020b1bd6e143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:25:00.432440 14863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 746f3456c10949c3b890020b1bd6e143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:25:00.432574 15003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 746f3456c10949c3b890020b1bd6e143. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:25:00.434943 15098 tablet_bootstrap.cc:492] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25: Bootstrap starting.
I20260619 12:25:00.435055 15097 tablet_bootstrap.cc:492] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b: Bootstrap starting.
I20260619 12:25:00.435827 15098 tablet_bootstrap.cc:654] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25: Neither blocks nor log segments found. Creating new log.
I20260619 12:25:00.435871 15097 tablet_bootstrap.cc:654] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b: Neither blocks nor log segments found. Creating new log.
I20260619 12:25:00.436941 15100 tablet_bootstrap.cc:492] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6: Bootstrap starting.
I20260619 12:25:00.437139 15097 tablet_bootstrap.cc:492] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b: No bootstrap required, opened a new log
I20260619 12:25:00.437222 15097 ts_tablet_manager.cc:1397] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260619 12:25:00.437628 15097 raft_consensus.cc:348] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.437744 15097 raft_consensus.cc:374] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:25:00.437773 15097 raft_consensus.cc:729] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c3c56beeca344caa27c3cfe57e57a0b, State: Initialized, Role: FOLLOWER
I20260619 12:25:00.437811 15100 tablet_bootstrap.cc:654] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6: Neither blocks nor log segments found. Creating new log.
I20260619 12:25:00.437891 15097 consensus_queue.cc:260] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.438128 15097 ts_tablet_manager.cc:1428] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260619 12:25:00.439277 15098 tablet_bootstrap.cc:492] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25: No bootstrap required, opened a new log
I20260619 12:25:00.439368 15098 ts_tablet_manager.cc:1397] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260619 12:25:00.439841 15100 tablet_bootstrap.cc:492] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6: No bootstrap required, opened a new log
I20260619 12:25:00.439839 15098 raft_consensus.cc:348] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.439917 15100 ts_tablet_manager.cc:1397] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 12:25:00.439951 15098 raft_consensus.cc:374] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:25:00.439983 15098 raft_consensus.cc:729] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 657d8ebe44b74eb6952d70a0d44fbb25, State: Initialized, Role: FOLLOWER
I20260619 12:25:00.440111 15098 consensus_queue.cc:260] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.440286 15100 raft_consensus.cc:348] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.440384 15100 raft_consensus.cc:374] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:25:00.440400 15098 ts_tablet_manager.cc:1428] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:25:00.440411 15100 raft_consensus.cc:729] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1007bb3e7e554133beb50e603f7f7aa6, State: Initialized, Role: FOLLOWER
I20260619 12:25:00.440544 15100 consensus_queue.cc:260] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.440778 15100 ts_tablet_manager.cc:1428] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:25:00.463574 15104 raft_consensus.cc:482] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:25:00.463704 15104 raft_consensus.cc:504] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.464036 15104 leader_election.cc:288] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193:36217), 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195:34767)
I20260619 12:25:00.464504 15022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "746f3456c10949c3b890020b1bd6e143" candidate_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" is_pre_election: true
I20260619 12:25:00.464568 14743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "746f3456c10949c3b890020b1bd6e143" candidate_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1007bb3e7e554133beb50e603f7f7aa6" is_pre_election: true
I20260619 12:25:00.464643 15022 raft_consensus.cc:2454] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657d8ebe44b74eb6952d70a0d44fbb25 in term 0.
I20260619 12:25:00.464792 14743 raft_consensus.cc:2454] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 657d8ebe44b74eb6952d70a0d44fbb25 in term 0.
I20260619 12:25:00.465035 14815 leader_election.cc:302] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3c56beeca344caa27c3cfe57e57a0b, 657d8ebe44b74eb6952d70a0d44fbb25; no voters:
I20260619 12:25:00.465242 15104 raft_consensus.cc:2781] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:25:00.465301 15104 raft_consensus.cc:482] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:25:00.465328 15104 raft_consensus.cc:3037] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:00.466321 15104 raft_consensus.cc:504] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.466635 15104 leader_election.cc:288] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [CANDIDATE]: Term 1 election: Requested vote from peers 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193:36217), 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195:34767)
I20260619 12:25:00.466996 15022 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "746f3456c10949c3b890020b1bd6e143" candidate_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2c3c56beeca344caa27c3cfe57e57a0b"
I20260619 12:25:00.467017 14743 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "746f3456c10949c3b890020b1bd6e143" candidate_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1007bb3e7e554133beb50e603f7f7aa6"
I20260619 12:25:00.467119 15022 raft_consensus.cc:3037] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:00.467149 14743 raft_consensus.cc:3037] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:00.468041 15022 raft_consensus.cc:2454] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657d8ebe44b74eb6952d70a0d44fbb25 in term 1.
I20260619 12:25:00.468142 14743 raft_consensus.cc:2454] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 657d8ebe44b74eb6952d70a0d44fbb25 in term 1.
I20260619 12:25:00.468380 14815 leader_election.cc:302] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2c3c56beeca344caa27c3cfe57e57a0b, 657d8ebe44b74eb6952d70a0d44fbb25; no voters:
I20260619 12:25:00.468541 15104 raft_consensus.cc:2781] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:25:00.468649 15104 raft_consensus.cc:686] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 LEADER]: Becoming Leader. State: Replica: 657d8ebe44b74eb6952d70a0d44fbb25, State: Running, Role: LEADER
I20260619 12:25:00.468796 15104 consensus_queue.cc:237] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } }
I20260619 12:25:00.470356 14583 catalog_manager.cc:5495] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 reported cstate change: term changed from 0 to 1, leader changed from <none> to 657d8ebe44b74eb6952d70a0d44fbb25 (127.6.231.194). New cstate: current_term: 1 leader_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "657d8ebe44b74eb6952d70a0d44fbb25" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35953 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 } health_report { overall_health: UNKNOWN } } }
I20260619 12:25:00.493593 15022 raft_consensus.cc:1261] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Refusing update from remote peer 657d8ebe44b74eb6952d70a0d44fbb25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:25:00.493625 14743 raft_consensus.cc:1261] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Refusing update from remote peer 657d8ebe44b74eb6952d70a0d44fbb25: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:25:00.494019 15156 consensus_queue.cc:1035] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1007bb3e7e554133beb50e603f7f7aa6" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:25:00.494190 15104 consensus_queue.cc:1035] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2c3c56beeca344caa27c3cfe57e57a0b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 34767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:25:00.527755 14583 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:25:00.527948 14583 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46762:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:25:00.529989 14583 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c69cc20937fc417bb08ff839795597d6: Sending DeleteTablet for 3 replicas of tablet 746f3456c10949c3b890020b1bd6e143
I20260619 12:25:00.530643 14723 tablet_service.cc:1514] Processing DeleteTablet for tablet 746f3456c10949c3b890020b1bd6e143 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:25:00 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:39370
I20260619 12:25:00.530682 15003 tablet_service.cc:1514] Processing DeleteTablet for tablet 746f3456c10949c3b890020b1bd6e143 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:25:00 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:37112
I20260619 12:25:00.530695 14863 tablet_service.cc:1514] Processing DeleteTablet for tablet 746f3456c10949c3b890020b1bd6e143 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:25:00 UTC) from {username='oryx', principal='oryx/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:39614
I20260619 12:25:00.530874 15162 tablet_replica.cc:330] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6: stopping tablet replica
I20260619 12:25:00.530874 15161 tablet_replica.cc:330] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b: stopping tablet replica
I20260619 12:25:00.530884 15163 tablet_replica.cc:330] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25: stopping tablet replica
I20260619 12:25:00.531000 15163 raft_consensus.cc:2229] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:25:00.530993 15162 raft_consensus.cc:2229] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:25:00.530998 15161 raft_consensus.cc:2229] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:25:00.531119 15161 raft_consensus.cc:2258] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:25:00.531119 15162 raft_consensus.cc:2258] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:25:00.531157 15163 raft_consensus.cc:2258] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:25:00.531968 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 14658
I20260619 12:25:00.531960 15163 ts_tablet_manager.cc:1905] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:25:00.532181 15161 ts_tablet_manager.cc:1905] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:25:00.532181 15162 ts_tablet_manager.cc:1905] T 746f3456c10949c3b890020b1bd6e143 P 1007bb3e7e554133beb50e603f7f7aa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:25:00.534885 15163 ts_tablet_manager.cc:1918] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:25:00.534953 15163 log.cc:1196] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/746f3456c10949c3b890020b1bd6e143
I20260619 12:25:00.535107 15161 ts_tablet_manager.cc:1918] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260619 12:25:00.535218 15161 log.cc:1196] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/746f3456c10949c3b890020b1bd6e143
I20260619 12:25:00.535209 15163 ts_tablet_manager.cc:1939] T 746f3456c10949c3b890020b1bd6e143 P 657d8ebe44b74eb6952d70a0d44fbb25: Deleting consensus metadata
I20260619 12:25:00.535470 15161 ts_tablet_manager.cc:1939] T 746f3456c10949c3b890020b1bd6e143 P 2c3c56beeca344caa27c3cfe57e57a0b: Deleting consensus metadata
I20260619 12:25:00.536125 14572 catalog_manager.cc:4841] TS 657d8ebe44b74eb6952d70a0d44fbb25 (127.6.231.194:35953): tablet 746f3456c10949c3b890020b1bd6e143 (table test-table [id=63ebb53627584b9eb28e9578110b748b]) successfully deleted
I20260619 12:25:00.537218 14571 catalog_manager.cc:4841] TS 2c3c56beeca344caa27c3cfe57e57a0b (127.6.231.195:34767): tablet 746f3456c10949c3b890020b1bd6e143 (table test-table [id=63ebb53627584b9eb28e9578110b748b]) successfully deleted
W20260619 12:25:00.543047 14572 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:36217 (error 108)
I20260619 12:25:00.543239 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 14798
I20260619 12:25:00.545289 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:00.544781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:36217 (local address 127.0.0.1:39394)
0619 12:25:00.545054 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:00.545149 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:36217: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":65,"threads_started":1}
W20260619 12:25:00.545524 14572 catalog_manager.cc:4584] TS 1007bb3e7e554133beb50e603f7f7aa6 (127.6.231.193:36217): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 746f3456c10949c3b890020b1bd6e143: Network error: Client connection negotiation failed: client connection to 127.6.231.193:36217: connect: Connection refused (error 111)
I20260619 12:25:00.550468 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 14938
I20260619 12:25:00.556955 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 14554
2026-06-19T12:25:00Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3267 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:25:00 dist-test-slave-kr9h krb5kdc[15195](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:25:00 dist-test-slave-kr9h krb5kdc[15195](info): set up 2 sockets
Jun 19 12:25:00 dist-test-slave-kr9h krb5kdc[15195](info): commencing operation
krb5kdc: starting...
W20260619 12:25:01.606215 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:25:01 dist-test-slave-kr9h krb5kdc[15195](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871901, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:25:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:25:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:25:01.730758 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:42355
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:32961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:42355
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:25:01.840296 15208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:01.840585 15208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:01.840626 15208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:01.844578 15208 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:25:01.844637 15208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:01.844657 15208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:01.844673 15208 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:25:01.844688 15208 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:25:01.850070 15208 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:32961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:42355
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:42355
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:01.851089 15208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:01.852037 15208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:01.860632 15216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:01.860682 15214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:01.860689 15213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:01.860792 15208 server_base.cc:1034] running on GCE node
I20260619 12:25:01.861608 15208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:01.862742 15208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:01.863950 15208 hybrid_clock.cc:648] HybridClock initialized: now 1781871901863929 us; error 35 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-06-19T12:25:01Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1309 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:25:01 dist-test-slave-kr9h krb5kdc[15221](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:25:01 dist-test-slave-kr9h krb5kdc[15221](info): set up 2 sockets
Jun 19 12:25:01 dist-test-slave-kr9h krb5kdc[15221](info): commencing operation
krb5kdc: starting...
W20260619 12:25:02.930420 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:25:03 dist-test-slave-kr9h krb5kdc[15221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:25:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:25:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:25:03.054596 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:35797
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:35797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:25:03.166781 15234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:03.167096 15234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:03.167182 15234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:03.171200 15234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:25:03.171296 15234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:03.171356 15234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:03.171416 15234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:25:03.171455 15234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:25:03.176817 15234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:35797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:35797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:03.177922 15234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:03.178979 15234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:25:03.187582 15234 server_base.cc:1034] running on GCE node
W20260619 12:25:03.187525 15242 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:03.187533 15239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:03.187528 15240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:03.188346 15234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:03.189563 15234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:03.190744 15234 hybrid_clock.cc:648] HybridClock initialized: now 1781871903190715 us; error 49 us; skew 500 ppm
Jun 19 12:25:03 dist-test-slave-kr9h krb5kdc[15221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871903, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:25:03.193570 15234 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:25:03.194439 15234 webserver.cc:460] Webserver started at http://127.6.231.254:46875/ using document root <none> and password file <none>
I20260619 12:25:03.195021 15234 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:03.195115 15234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:03.195324 15234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:03.197381 15234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "20a1be23d4564f4a98746be938abb633"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "04b2788656688201c9fe2253d6313479"
server_key_iv: "a027142407e62c1b0aa6fd8498c2786a"
server_key_version: "encryptionkey@0"
I20260619 12:25:03.198000 15234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "20a1be23d4564f4a98746be938abb633"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "04b2788656688201c9fe2253d6313479"
server_key_iv: "a027142407e62c1b0aa6fd8498c2786a"
server_key_version: "encryptionkey@0"
I20260619 12:25:03.201687 15234 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:25:03.204022 15249 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:03.205268 15234 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:25:03.205399 15234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "20a1be23d4564f4a98746be938abb633"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "04b2788656688201c9fe2253d6313479"
server_key_iv: "a027142407e62c1b0aa6fd8498c2786a"
server_key_version: "encryptionkey@0"
I20260619 12:25:03.205577 15234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:03.222823 15234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:03.226289 15234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:03.226507 15234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:03.233969 15234 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:35797
I20260619 12:25:03.234019 15301 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:35797 every 8 connection(s)
I20260619 12:25:03.235368 15234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:25:03.240016 15302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:25:03.240885 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 15234
I20260619 12:25:03.241011 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:25:03.241346 7071 external_mini_cluster.cc:1428] Setting key 2e9852ac7c42a82be3d40879fc1b1e53
I20260619 12:25:03.246158 15302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633: Bootstrap starting.
I20260619 12:25:03.248337 15302 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633: Neither blocks nor log segments found. Creating new log.
I20260619 12:25:03.249070 15302 log.cc:824] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633: Log is configured to *not* fsync() on all Append() calls
Jun 19 12:25:03 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:25:03.251015 15302 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633: No bootstrap required, opened a new log
I20260619 12:25:03.254534 15302 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a1be23d4564f4a98746be938abb633" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35797 } }
I20260619 12:25:03.254855 15302 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:25:03.254935 15302 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20a1be23d4564f4a98746be938abb633, State: Initialized, Role: FOLLOWER
I20260619 12:25:03.255414 15302 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a1be23d4564f4a98746be938abb633" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35797 } }
I20260619 12:25:03.255566 15302 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:25:03.255636 15302 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:25:03.255730 15302 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:03.255736 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:03.242868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33404 (local address 127.6.231.254:35797)
0619 12:25:03.243348 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:03.243364 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:03.243400 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:25:03.244032 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:03.244049 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:03.244395 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:03.244723 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:03.245599 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.246637 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.247508 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.247980 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.248154 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:03.250707 (+ 2553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:03.250713 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:03.250727 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:25:03.250753 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:03.252539 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:03.253091 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:03.253097 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:03.253109 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:03.253218 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:03.253634 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:03.253637 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:03.253639 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:03.254039 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:03.254256 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:03.254507 (+ 251us) server_negotiation.cc:299] Negotiation successful
0619 12:25:03.254803 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":136,"threads_started":1}
I20260619 12:25:03.256917 15302 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a1be23d4564f4a98746be938abb633" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35797 } }
I20260619 12:25:03.257263 15302 leader_election.cc:302] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20a1be23d4564f4a98746be938abb633; no voters:
I20260619 12:25:03.257625 15302 leader_election.cc:288] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:25:03.257696 15307 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:25:03.257911 15307 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 1 LEADER]: Becoming Leader. State: Replica: 20a1be23d4564f4a98746be938abb633, State: Running, Role: LEADER
I20260619 12:25:03.258555 15307 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a1be23d4564f4a98746be938abb633" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35797 } }
I20260619 12:25:03.258970 15302 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:25:03.259757 15307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20a1be23d4564f4a98746be938abb633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a1be23d4564f4a98746be938abb633" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35797 } } }
I20260619 12:25:03.259979 15307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [sys.catalog]: This master's current role is: LEADER
I20260619 12:25:03.260151 15308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20a1be23d4564f4a98746be938abb633. Latest consensus state: current_term: 1 leader_uuid: "20a1be23d4564f4a98746be938abb633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a1be23d4564f4a98746be938abb633" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35797 } } }
I20260619 12:25:03.260334 15308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [sys.catalog]: This master's current role is: LEADER
I20260619 12:25:03.260535 15316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:25:03.263286 15316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:25:03.268678 15316 catalog_manager.cc:1347] Generated new cluster ID: d8d9669e4a04457581526c9eef547d64
I20260619 12:25:03.268741 15316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:25:03.273108 15316 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:25:03.274358 15316 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:25:03.281513 15316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633: Generated new TSK 0
I20260619 12:25:03.282316 15316 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:25:03.318645 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35797
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:25:03.433446 15328 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:03.433727 15328 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:03.433770 15328 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:03.438017 15328 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:03.438092 15328 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:03.438175 15328 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:25:03.443614 15328 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15328
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:03.444782 15328 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:03.445891 15328 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:03.455958 15336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:03.455955 15333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:03.455982 15334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:03.456147 15328 server_base.cc:1034] running on GCE node
I20260619 12:25:03.456704 15328 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:03.457486 15328 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:03.458683 15328 hybrid_clock.cc:648] HybridClock initialized: now 1781871903458660 us; error 38 us; skew 500 ppm
Jun 19 12:25:03 dist-test-slave-kr9h krb5kdc[15221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871903, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:25:03.461948 15328 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:25:03.462954 15328 webserver.cc:460] Webserver started at http://127.6.231.193:42347/ using document root <none> and password file <none>
I20260619 12:25:03.463578 15328 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:03.463685 15328 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:03.463943 15328 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:03.466349 15328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "911238431228474c9ab485adfefd553c"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "e11069fa1423ecd67f916be2c81c0ff8"
server_key_iv: "b609e96b82fe077124bf71e70aef0bd1"
server_key_version: "encryptionkey@0"
I20260619 12:25:03.466969 15328 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "911238431228474c9ab485adfefd553c"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "e11069fa1423ecd67f916be2c81c0ff8"
server_key_iv: "b609e96b82fe077124bf71e70aef0bd1"
server_key_version: "encryptionkey@0"
I20260619 12:25:03.471050 15328 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:25:03.473801 15343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:03.475227 15328 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 12:25:03.475402 15328 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "911238431228474c9ab485adfefd553c"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "e11069fa1423ecd67f916be2c81c0ff8"
server_key_iv: "b609e96b82fe077124bf71e70aef0bd1"
server_key_version: "encryptionkey@0"
I20260619 12:25:03.475526 15328 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:03.487757 15328 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:03.491274 15328 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:03.491496 15328 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:03.492110 15328 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:03.492960 15328 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:03.493031 15328 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:03.493103 15328 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:03.493146 15328 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:03.502905 15328 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:46259
I20260619 12:25:03.502925 15456 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:46259 every 8 connection(s)
I20260619 12:25:03.503822 15328 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:25:03.505412 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 15328
I20260619 12:25:03.505606 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:25:03.505952 7071 external_mini_cluster.cc:1428] Setting key cb3a43d03e09c6fc55bb41c8e23625d2
Jun 19 12:25:03 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871903, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:25:03.520555 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:03.505748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:43357 (local address 127.6.231.254:35797)
0619 12:25:03.505928 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:03.505933 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:03.506962 (+ 1029us) server_negotiation.cc:407] Connection header received
0619 12:25:03.508236 (+ 1274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:03.508238 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:03.508287 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:03.508381 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:03.510072 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.510636 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.512584 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.512826 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.512923 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:03.516084 (+ 3161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:03.516087 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:03.516089 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:25:03.516115 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:03.517936 (+ 1821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:03.518677 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:03.518684 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:03.518690 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:03.518814 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:03.519247 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:03.519251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:03.519253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:03.519513 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:03.519638 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:03.520210 (+ 572us) server_negotiation.cc:299] Negotiation successful
0619 12:25:03.520366 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 12:25:03.521536 15459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:03.506293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:43357)
0619 12:25:03.506795 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:03.506856 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:03.508029 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:03.508575 (+ 546us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:03.508584 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:03.509130 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:03.509874 (+ 744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:03.509896 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.510776 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.510783 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:03.512406 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:03.512422 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.513004 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.513011 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:03.513254 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:03.513973 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:03.513996 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:03.515893 (+ 1897us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:25:03.518091 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:25:03.518097 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:25:03.518113 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:25:03.518544 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:25:03.518964 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:25:03.518967 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:25:03.518968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:25:03.519125 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:25:03.519634 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:25:03.519642 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:25:03.519922 (+ 280us) client_negotiation.cc:769] Sending connection context
0619 12:25:03.520167 (+ 245us) client_negotiation.cc:240] Negotiation successful
0619 12:25:03.520543 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":121,"threads_started":1}
I20260619 12:25:03.522982 15457 heartbeater.cc:347] Connected to a master server at 127.6.231.254:35797
I20260619 12:25:03.523240 15457 heartbeater.cc:464] Registering TS with master...
I20260619 12:25:03.523775 15457 heartbeater.cc:511] Master 127.6.231.254:35797 requested a full tablet report, sending...
I20260619 12:25:03.525301 15266 ts_manager.cc:194] Registered new tserver with Master: 911238431228474c9ab485adfefd553c (127.6.231.193:46259)
I20260619 12:25:03.526517 15266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:43357
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:25:03.537032 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35797
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:25:03.653240 15462 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:03.653566 15462 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:03.653632 15462 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:03.657785 15462 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:03.657856 15462 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:03.657917 15462 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:25:03.663779 15462 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15462
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:03.664896 15462 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:03.666083 15462 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:03.676402 15468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:03.676402 15470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:03.676576 15462 server_base.cc:1034] running on GCE node
W20260619 12:25:03.676494 15467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:03.677109 15462 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:03.677898 15462 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:03.679071 15462 hybrid_clock.cc:648] HybridClock initialized: now 1781871903679008 us; error 76 us; skew 500 ppm
Jun 19 12:25:03 dist-test-slave-kr9h krb5kdc[15221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871903, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:25:03.681967 15462 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:25:03.682826 15462 webserver.cc:460] Webserver started at http://127.6.231.194:46057/ using document root <none> and password file <none>
I20260619 12:25:03.683424 15462 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:03.683483 15462 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:03.683691 15462 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:03.686204 15462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "6304ce852eb44cac84495d037df1dd40"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "b40a21d71d6a6191ad8e33e3947372a1"
server_key_iv: "bf5f3b306aa48011789e1c6dc3114a11"
server_key_version: "encryptionkey@0"
I20260619 12:25:03.686764 15462 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "6304ce852eb44cac84495d037df1dd40"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "b40a21d71d6a6191ad8e33e3947372a1"
server_key_iv: "bf5f3b306aa48011789e1c6dc3114a11"
server_key_version: "encryptionkey@0"
I20260619 12:25:03.690757 15462 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260619 12:25:03.693569 15477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:03.695030 15462 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 12:25:03.695142 15462 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "6304ce852eb44cac84495d037df1dd40"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "b40a21d71d6a6191ad8e33e3947372a1"
server_key_iv: "bf5f3b306aa48011789e1c6dc3114a11"
server_key_version: "encryptionkey@0"
I20260619 12:25:03.695307 15462 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:03.710618 15462 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:03.713946 15462 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:03.714154 15462 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:03.714838 15462 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:03.715693 15462 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:03.715740 15462 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:03.715816 15462 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:03.715849 15462 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:03.724839 15462 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:45575
I20260619 12:25:03.724948 15590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:45575 every 8 connection(s)
I20260619 12:25:03.725863 15462 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:25:03.734387 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 15462
I20260619 12:25:03.734541 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:25:03.734935 7071 external_mini_cluster.cc:1428] Setting key 9e200bfd37404bbb87a419c9be59588b
Jun 19 12:25:03 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871903, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:25:03.742110 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:03.727736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:47629 (local address 127.6.231.254:35797)
0619 12:25:03.727884 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:03.727887 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:03.728877 (+ 990us) server_negotiation.cc:407] Connection header received
0619 12:25:03.730336 (+ 1459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:03.730339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:03.730390 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:03.730472 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:03.731867 (+ 1395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.732441 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.733760 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.733999 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.734085 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:03.737499 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:03.737503 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:03.737505 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:25:03.737531 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:03.739256 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:03.740015 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:03.740020 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:03.740025 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:03.740121 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:03.740583 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:03.740590 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:03.740594 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:03.740870 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:03.740981 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:03.741756 (+ 775us) server_negotiation.cc:299] Negotiation successful
0619 12:25:03.741911 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 12:25:03.742936 15593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:03.728151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:47629)
0619 12:25:03.728686 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:03.728759 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:03.730044 (+ 1285us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:03.730684 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:03.730692 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:03.731079 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:03.731659 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:03.731673 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.732571 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.732574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:03.733637 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:03.733647 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.734187 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.734195 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:03.734519 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:03.735342 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:03.735364 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:03.737169 (+ 1805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:25:03.739440 (+ 2271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:25:03.739448 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:25:03.739472 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:25:03.739885 (+ 413us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:25:03.740265 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:25:03.740268 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:25:03.740309 (+ 41us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:25:03.740473 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:25:03.741005 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:25:03.741012 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:25:03.741410 (+ 398us) client_negotiation.cc:769] Sending connection context
0619 12:25:03.741673 (+ 263us) client_negotiation.cc:240] Negotiation successful
0619 12:25:03.741976 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":110,"threads_started":1}
I20260619 12:25:03.744194 15591 heartbeater.cc:347] Connected to a master server at 127.6.231.254:35797
I20260619 12:25:03.744448 15591 heartbeater.cc:464] Registering TS with master...
I20260619 12:25:03.744968 15591 heartbeater.cc:511] Master 127.6.231.254:35797 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
I20260619 12:25:03.746111 15266 ts_manager.cc:194] Registered new tserver with Master: 6304ce852eb44cac84495d037df1dd40 (127.6.231.194:45575)
I20260619 12:25:03.746742 15266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:47629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:25:03.765383 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35797
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:25:03.880084 15596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:03.880430 15596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:03.880498 15596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:03.884887 15596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:03.884970 15596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:03.885032 15596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:25:03.890930 15596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15596
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:03.892143 15596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:03.893282 15596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:03.903458 15604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:03.903538 15602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:03.903461 15601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:03.903797 15596 server_base.cc:1034] running on GCE node
I20260619 12:25:03.904287 15596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:03.904985 15596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:03.906144 15596 hybrid_clock.cc:648] HybridClock initialized: now 1781871903906126 us; error 39 us; skew 500 ppm
Jun 19 12:25:03 dist-test-slave-kr9h krb5kdc[15221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871903, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:25:03.909099 15596 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:25:03.910017 15596 webserver.cc:460] Webserver started at http://127.6.231.195:36481/ using document root <none> and password file <none>
I20260619 12:25:03.910557 15596 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:03.910627 15596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:03.910806 15596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:03.912817 15596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "0c4220c9523441a8af7fca4c0ea51f31"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "b3c0425a5a4235dc6463c922150ca9ee"
server_key_iv: "1e4bb1b15598b43c972ad4bfb48eebdf"
server_key_version: "encryptionkey@0"
I20260619 12:25:03.913331 15596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "0c4220c9523441a8af7fca4c0ea51f31"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "b3c0425a5a4235dc6463c922150ca9ee"
server_key_iv: "1e4bb1b15598b43c972ad4bfb48eebdf"
server_key_version: "encryptionkey@0"
I20260619 12:25:03.917155 15596 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260619 12:25:03.919659 15611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:03.920976 15596 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:25:03.921123 15596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "0c4220c9523441a8af7fca4c0ea51f31"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "b3c0425a5a4235dc6463c922150ca9ee"
server_key_iv: "1e4bb1b15598b43c972ad4bfb48eebdf"
server_key_version: "encryptionkey@0"
I20260619 12:25:03.921244 15596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:03.938509 15596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:03.942039 15596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:03.942269 15596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:03.942862 15596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:03.943733 15596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:03.943779 15596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:03.943847 15596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:03.943883 15596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:03.952298 15596 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:39837
I20260619 12:25:03.952337 15724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:39837 every 8 connection(s)
I20260619 12:25:03.953209 15596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:25:03.962067 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 15596
I20260619 12:25:03.962179 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:25:03.962498 7071 external_mini_cluster.cc:1428] Setting key 99ea687070681ff64e49e3083f2683c4
Jun 19 12:25:03 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871903, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:25:03.967653 15305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:03.955055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:47875 (local address 127.6.231.254:35797)
0619 12:25:03.955196 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:03.955200 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:03.956068 (+ 868us) server_negotiation.cc:407] Connection header received
0619 12:25:03.957385 (+ 1317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:03.957389 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:03.957467 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:03.957544 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:03.958768 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.959328 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.960644 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.960876 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.960989 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:03.963869 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:03.963873 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:03.963877 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:25:03.963909 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:03.965519 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:03.966067 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:03.966071 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:03.966073 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:03.966157 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:03.966524 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:03.966529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:03.966531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:03.966736 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:03.966847 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:03.967364 (+ 517us) server_negotiation.cc:299] Negotiation successful
0619 12:25:03.967490 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:25:03.968611 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:03.955474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:47875)
0619 12:25:03.955910 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:03.955952 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:03.957156 (+ 1204us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:03.957720 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:03.957727 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:03.958086 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:03.958594 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:03.958607 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.959451 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.959455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:03.960515 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:03.960524 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:03.961027 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:03.961031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:03.961199 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:03.961863 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:03.961879 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:03.963681 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:25:03.965650 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:25:03.965656 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:25:03.965672 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:25:03.965959 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:25:03.966266 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:25:03.966270 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:25:03.966272 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:25:03.966426 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:25:03.966845 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:25:03.966852 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:25:03.967124 (+ 272us) client_negotiation.cc:769] Sending connection context
0619 12:25:03.967325 (+ 201us) client_negotiation.cc:240] Negotiation successful
0619 12:25:03.967675 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":101,"threads_started":1}
I20260619 12:25:03.969973 15725 heartbeater.cc:347] Connected to a master server at 127.6.231.254:35797
I20260619 12:25:03.970325 15725 heartbeater.cc:464] Registering TS with master...
I20260619 12:25:03.970882 15725 heartbeater.cc:511] Master 127.6.231.254:35797 requested a full tablet report, sending...
I20260619 12:25:03.972016 15266 ts_manager.cc:194] Registered new tserver with Master: 0c4220c9523441a8af7fca4c0ea51f31 (127.6.231.195:39837)
I20260619 12:25:03.972674 15266 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:47875
I20260619 12:25:03.976846 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab.
Entry for principal oryx/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab.
WARNING: no policy specified for oryx/127.6.231.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.231.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab.
Entry for principal oryx/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab.
WARNING: no policy specified for oryx/127.6.231.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.6.231.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab.
Entry for principal oryx/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab.
I20260619 12:25:04.059984 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 15328
I20260619 12:25:04.065838 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 15462
I20260619 12:25:04.071137 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 15596
I20260619 12:25:04.076366 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 15234
I20260619 12:25:04.082487 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:35797
--webserver_interface=127.6.231.254
--webserver_port=46875
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:35797
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:25:04.197384 15734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:04.197739 15734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:04.197827 15734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:04.201967 15734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:25:04.202077 15734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:04.202111 15734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:04.202148 15734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:25:04.202179 15734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:25:04.207684 15734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:35797
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:35797
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=46875
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:04.208884 15734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:04.209983 15734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:04.219163 15740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:04.219152 15742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:04.219149 15739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:04.219468 15734 server_base.cc:1034] running on GCE node
I20260619 12:25:04.220085 15734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:04.221339 15734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:04.222569 15734 hybrid_clock.cc:648] HybridClock initialized: now 1781871904222538 us; error 50 us; skew 500 ppm
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871904, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:25:04.226009 15734 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:25:04.227013 15734 webserver.cc:460] Webserver started at http://127.6.231.254:46875/ using document root <none> and password file <none>
I20260619 12:25:04.227622 15734 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:04.227768 15734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:04.231909 15734 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:25:04.234438 15749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:04.235783 15734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:25:04.235934 15734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "20a1be23d4564f4a98746be938abb633"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "04b2788656688201c9fe2253d6313479"
server_key_iv: "a027142407e62c1b0aa6fd8498c2786a"
server_key_version: "encryptionkey@0"
I20260619 12:25:04.236538 15734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:04.255795 15734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:04.258881 15734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:04.259104 15734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:04.266261 15734 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:35797
I20260619 12:25:04.266290 15801 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:35797 every 8 connection(s)
I20260619 12:25:04.267375 15734 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:25:04.269482 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 15734
I20260619 12:25:04.269942 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:46259
--local_ip_for_outbound_sockets=127.6.231.193
--tserver_master_addrs=127.6.231.254:35797
--webserver_port=42347
--webserver_interface=127.6.231.193
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.231.193
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:25:04.272362 15802 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:25:04.274209 15802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633: Bootstrap starting.
I20260619 12:25:04.290848 15802 log.cc:824] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633: Log is configured to *not* fsync() on all Append() calls
I20260619 12:25:04.297367 15802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:25:04.297791 15802 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633: Bootstrap complete.
I20260619 12:25:04.302165 15802 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a1be23d4564f4a98746be938abb633" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35797 } }
I20260619 12:25:04.302609 15802 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20a1be23d4564f4a98746be938abb633, State: Initialized, Role: FOLLOWER
I20260619 12:25:04.303104 15802 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a1be23d4564f4a98746be938abb633" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35797 } }
I20260619 12:25:04.303262 15802 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:25:04.303364 15802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:25:04.303505 15802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:25:04.305593 15802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a1be23d4564f4a98746be938abb633" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35797 } }
I20260619 12:25:04.305925 15802 leader_election.cc:302] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20a1be23d4564f4a98746be938abb633; no voters:
I20260619 12:25:04.306257 15802 leader_election.cc:288] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 12:25:04.306342 15806 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 12:25:04.306545 15806 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [term 2 LEADER]: Becoming Leader. State: Replica: 20a1be23d4564f4a98746be938abb633, State: Running, Role: LEADER
I20260619 12:25:04.306836 15806 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a1be23d4564f4a98746be938abb633" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35797 } }
I20260619 12:25:04.307482 15802 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:25:04.307817 15808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20a1be23d4564f4a98746be938abb633. Latest consensus state: current_term: 2 leader_uuid: "20a1be23d4564f4a98746be938abb633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a1be23d4564f4a98746be938abb633" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35797 } } }
I20260619 12:25:04.308001 15808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [sys.catalog]: This master's current role is: LEADER
I20260619 12:25:04.308176 15807 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "20a1be23d4564f4a98746be938abb633" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20a1be23d4564f4a98746be938abb633" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35797 } } }
I20260619 12:25:04.308290 15807 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633 [sys.catalog]: This master's current role is: LEADER
I20260619 12:25:04.308524 15810 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:25:04.312538 15810 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:25:04.313203 15810 catalog_manager.cc:1259] Loaded cluster ID: d8d9669e4a04457581526c9eef547d64
I20260619 12:25:04.313267 15810 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:25:04.315644 15810 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:25:04.316282 15810 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 20a1be23d4564f4a98746be938abb633: Loaded TSK: 0
I20260619 12:25:04.317521 15810 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:25:04.388118 15804 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:04.388435 15804 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:04.388514 15804 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:04.392702 15804 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:04.392800 15804 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:04.392904 15804 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:25:04.398363 15804 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.193.keytab
--principal=oryx/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:46259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=42347
--tserver_master_addrs=127.6.231.254:35797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15804
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:04.399412 15804 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:04.400437 15804 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:04.410060 15829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:04.410035 15828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:04.410036 15831 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:04.410336 15804 server_base.cc:1034] running on GCE node
I20260619 12:25:04.411094 15804 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:04.411806 15804 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:04.413000 15804 hybrid_clock.cc:648] HybridClock initialized: now 1781871904412977 us; error 46 us; skew 500 ppm
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871904, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:25:04.416987 15804 init.cc:377] Logged in from keytab as oryx/127.6.231.193@KRBTEST.COM (short username oryx)
I20260619 12:25:04.418071 15804 webserver.cc:460] Webserver started at http://127.6.231.193:42347/ using document root <none> and password file <none>
I20260619 12:25:04.418673 15804 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:04.418781 15804 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:04.422760 15804 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260619 12:25:04.424962 15838 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:04.426343 15804 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 12:25:04.426471 15804 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "911238431228474c9ab485adfefd553c"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "e11069fa1423ecd67f916be2c81c0ff8"
server_key_iv: "b609e96b82fe077124bf71e70aef0bd1"
server_key_version: "encryptionkey@0"
I20260619 12:25:04.427042 15804 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:04.446944 15804 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:04.450344 15804 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:04.450556 15804 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:04.451236 15804 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:04.452150 15804 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:04.452224 15804 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:04.452293 15804 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:04.452345 15804 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:04.461491 15804 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:46259
I20260619 12:25:04.461529 15951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:46259 every 8 connection(s)
I20260619 12:25:04.462528 15804 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:25:04.466954 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 15804
I20260619 12:25:04.467388 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:45575
--local_ip_for_outbound_sockets=127.6.231.194
--tserver_master_addrs=127.6.231.254:35797
--webserver_port=46057
--webserver_interface=127.6.231.194
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.231.194
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:04.477558 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:04.464643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:37077)
0619 12:25:04.465105 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:04.465157 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:04.466377 (+ 1220us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:04.467661 (+ 1284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:04.467672 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:04.468209 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:04.468924 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.468942 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.470919 (+ 1977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.470921 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.471934 (+ 1013us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.471944 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.473126 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.473128 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.473274 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.474020 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:04.474037 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:04.476680 (+ 2643us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":143,"threads_started":1}
W20260619 12:25:04.478045 15954 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:04.464570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:37077 (local address 127.6.231.254:35797)
0619 12:25:04.465057 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:04.465074 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:04.465281 (+ 207us) server_negotiation.cc:407] Connection header received
0619 12:25:04.466825 (+ 1544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:04.466837 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:04.467219 (+ 382us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:04.467574 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:04.469121 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.470768 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.472067 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.472957 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.473163 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.477014 (+ 3851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:37077: BlockingRecv error: recv got EOF from 127.6.231.193:37077 (error 108)
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":148,"threads_started":1}
W20260619 12:25:04.478209 15952 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:35797 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.231.254:35797: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:04.484619 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:04.478766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:43721)
0619 12:25:04.478941 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:04.478957 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:04.479042 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:04.479281 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:04.479284 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:04.479466 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:04.479585 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.479591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.480325 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.480328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.481246 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.481256 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.481832 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.481836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.481931 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.482468 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:04.482482 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:04.484464 (+ 1982us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":73}
W20260619 12:25:04.484833 15954 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:04.478905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:43721 (local address 127.6.231.254:35797)
0619 12:25:04.479029 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:04.479032 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:04.479045 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:25:04.479134 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:04.479136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:04.479175 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:04.479242 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:04.479697 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.480222 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.481373 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.481728 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.481817 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.484593 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:43721: BlockingRecv error: recv got EOF from 127.6.231.193:43721 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:04.490604 15955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:04.485151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:34637)
0619 12:25:04.485370 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:04.485403 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:04.485518 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:04.485775 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:04.485778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:04.485940 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:04.486012 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.486017 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.486749 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.486752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.487323 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.487331 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.487901 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.487905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.487992 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.488571 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:04.488584 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:04.490441 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138}
W20260619 12:25:04.490861 15952 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:25:04.490991 15954 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:04.485272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:34637 (local address 127.6.231.254:35797)
0619 12:25:04.485495 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:04.485499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:04.485512 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:25:04.485628 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:04.485631 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:04.485679 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:04.485761 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:04.486124 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.486631 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.487447 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.487754 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.487899 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.490572 (+ 2673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:34637: BlockingRecv error: recv got EOF from 127.6.231.193:34637 (error 108)
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":129}
W20260619 12:25:04.587781 15956 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:04.588097 15956 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:04.588181 15956 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:04.592479 15956 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:04.592582 15956 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:04.592680 15956 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:25:04.598483 15956 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.194.keytab
--principal=oryx/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:45575
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=46057
--tserver_master_addrs=127.6.231.254:35797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15956
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:04.599660 15956 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:04.600775 15956 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:04.611109 15961 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:04.611123 15964 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:04.611358 15962 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:04.611500 15956 server_base.cc:1034] running on GCE node
I20260619 12:25:04.611955 15956 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:04.612671 15956 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:04.613888 15956 hybrid_clock.cc:648] HybridClock initialized: now 1781871904613854 us; error 59 us; skew 500 ppm
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871904, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:25:04.616843 15956 init.cc:377] Logged in from keytab as oryx/127.6.231.194@KRBTEST.COM (short username oryx)
I20260619 12:25:04.617898 15956 webserver.cc:460] Webserver started at http://127.6.231.194:46057/ using document root <none> and password file <none>
I20260619 12:25:04.618530 15956 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:04.618645 15956 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:04.622692 15956 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:25:04.624979 15971 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:04.626439 15956 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:25:04.626590 15956 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "6304ce852eb44cac84495d037df1dd40"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "b40a21d71d6a6191ad8e33e3947372a1"
server_key_iv: "bf5f3b306aa48011789e1c6dc3114a11"
server_key_version: "encryptionkey@0"
I20260619 12:25:04.627157 15956 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:04.644227 15956 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:04.647439 15956 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:04.647645 15956 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:04.648258 15956 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:04.649127 15956 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:04.649197 15956 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:04.649266 15956 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:04.649318 15956 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:04.658929 15956 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:45575
I20260619 12:25:04.658946 16084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:45575 every 8 connection(s)
I20260619 12:25:04.659998 15956 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:25:04.665092 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 15956
I20260619 12:25:04.665623 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:39837
--local_ip_for_outbound_sockets=127.6.231.195
--tserver_master_addrs=127.6.231.254:35797
--webserver_port=36481
--webserver_interface=127.6.231.195
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.6.231.195
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 12:25:04.672370 15954 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:04.661915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:34693 (local address 127.6.231.254:35797)
0619 12:25:04.662086 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:04.662090 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:04.662968 (+ 878us) server_negotiation.cc:407] Connection header received
0619 12:25:04.664333 (+ 1365us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:04.664337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:04.664403 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:04.664519 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:04.666433 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.666971 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.668214 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.668466 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.668589 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.672175 (+ 3586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:34693: BlockingRecv error: recv got EOF from 127.6.231.194:34693 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 12:25:04.672999 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:04.662328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:34693)
0619 12:25:04.662803 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:04.662847 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:04.664047 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:04.664710 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:04.664717 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:04.665284 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:04.666062 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.666088 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.667139 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.667143 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.668072 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.668081 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.668631 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.668635 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.668828 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.669555 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:04.669572 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:04.672041 (+ 2469us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":112,"threads_started":1}
W20260619 12:25:04.673574 16085 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:35797 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.231.254:35797: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:04.679414 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:04.674077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:36497)
0619 12:25:04.674332 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:04.674344 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:04.674449 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:04.674694 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:04.674697 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:04.674872 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:04.674958 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.674963 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.675674 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.675681 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.676241 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.676255 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.676702 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.676706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.676800 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.677362 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:04.677373 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:04.679284 (+ 1911us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 12:25:04.679633 15954 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:04.674139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:36497 (local address 127.6.231.254:35797)
0619 12:25:04.674236 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:04.674240 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:04.674408 (+ 168us) server_negotiation.cc:407] Connection header received
0619 12:25:04.674544 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:04.674547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:04.674597 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:04.674680 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:04.675070 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.675570 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.676367 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.676589 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.676698 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.679511 (+ 2813us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:36497: BlockingRecv error: recv got EOF from 127.6.231.194:36497 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:04.685501 16087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:04.679943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:57417)
0619 12:25:04.680089 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:04.680102 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:04.680184 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:04.680422 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:04.680426 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:04.680584 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:04.680671 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.680676 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.681355 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.681359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.682319 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.682329 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.682809 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.682813 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.682896 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.683381 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:04.683395 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:04.685331 (+ 1936us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":89}
W20260619 12:25:04.685576 15954 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:04.680032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:57417 (local address 127.6.231.254:35797)
0619 12:25:04.680149 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:04.680152 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:04.680164 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:25:04.680284 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:04.680287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:04.680320 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:04.680384 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:04.680787 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.681259 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.682453 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.682703 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.682791 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.685463 (+ 2672us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:57417: BlockingRecv error: recv got EOF from 127.6.231.194:57417 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
W20260619 12:25:04.685928 16085 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:25:04.781279 16088 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:04.781628 16088 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:04.781706 16088 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:04.785768 16088 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:04.785863 16088 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:04.785955 16088 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:25:04.791435 16088 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/krb5kdc/oryx_127.6.231.195.keytab
--principal=oryx/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:39837
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=36481
--tserver_master_addrs=127.6.231.254:35797
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16088
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:04.792502 16088 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:04.793557 16088 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:04.803339 16096 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:04.803515 16088 server_base.cc:1034] running on GCE node
W20260619 12:25:04.803350 16093 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:04.803730 16094 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:04.804126 16088 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:04.804819 16088 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:04.805985 16088 hybrid_clock.cc:648] HybridClock initialized: now 1781871904805969 us; error 32 us; skew 500 ppm
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871904, etypes {rep=17 tkt=17 ses=17}, oryx/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:25:04.809324 16088 init.cc:377] Logged in from keytab as oryx/127.6.231.195@KRBTEST.COM (short username oryx)
I20260619 12:25:04.810300 16088 webserver.cc:460] Webserver started at http://127.6.231.195:36481/ using document root <none> and password file <none>
I20260619 12:25:04.810875 16088 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:04.810949 16088 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:04.815383 16088 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:25:04.817842 16103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:04.819301 16088 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 12:25:04.819414 16088 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "0c4220c9523441a8af7fca4c0ea51f31"
format_stamp: "Formatted at 2026-06-19 12:25:03 on dist-test-slave-kr9h"
server_key: "b3c0425a5a4235dc6463c922150ca9ee"
server_key_iv: "1e4bb1b15598b43c972ad4bfb48eebdf"
server_key_version: "encryptionkey@0"
I20260619 12:25:04.820004 16088 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:04.839572 16088 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:04.843297 16088 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:04.843518 16088 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:04.844200 16088 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:04.845057 16088 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:04.845103 16088 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:04.845165 16088 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:04.845198 16088 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:04.854684 16088 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:39837
I20260619 12:25:04.854707 16216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:39837 every 8 connection(s)
I20260619 12:25:04.855792 16088 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:25:04.862517 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 16088
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
W20260619 12:25:04.867517 15954 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:04.857725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:46213 (local address 127.6.231.254:35797)
0619 12:25:04.857888 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:04.857892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:04.858748 (+ 856us) server_negotiation.cc:407] Connection header received
0619 12:25:04.860142 (+ 1394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:04.860146 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:04.860200 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:04.860313 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:04.861537 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.862069 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.863391 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.863662 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.863755 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.867408 (+ 3653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:46213: BlockingRecv error: recv got EOF from 127.6.231.195:46213 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 12:25:04.868388 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:04.858104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:46213)
0619 12:25:04.858579 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:04.858622 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:04.859912 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:04.860494 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:04.860501 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:04.860864 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:04.861348 (+ 484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.861361 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.862203 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.862207 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.863254 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.863264 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.863954 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.863958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.864184 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.864808 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:04.864824 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:04.867279 (+ 2455us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":132,"threads_started":1}
W20260619 12:25:04.868947 16217 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:35797 (0 consecutive failures): Not authorized: Failed to ping master at 127.6.231.254:35797: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
I20260619 12:25:04.873221 16221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:04.863577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33414 (local address 127.6.231.254:35797)
0619 12:25:04.863870 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:04.863874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:04.863887 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:25:04.863920 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:04.863922 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:04.863962 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:04.864048 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:04.864768 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.865275 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.866275 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.866642 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.866752 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.868494 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:04.868504 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:04.868527 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:25:04.868566 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:04.870743 (+ 2177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:04.871235 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:04.871241 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:04.871247 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:04.871451 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:04.871797 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:04.871803 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:04.871807 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:04.872212 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:04.872418 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:04.872830 (+ 412us) server_negotiation.cc:299] Negotiation successful
0619 12:25:04.873086 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":114,"threads_started":1}
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:04.874915 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:04.869401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:50625)
0619 12:25:04.869607 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:04.869618 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:04.869707 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:04.869974 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:04.869979 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:04.870126 (+ 147us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:04.870237 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.870243 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.871031 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.871034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.871765 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.871778 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.872244 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.872250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.872349 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.872824 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:04.872842 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:04.874815 (+ 1973us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137}
W20260619 12:25:04.875072 15954 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:04.869520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:50625 (local address 127.6.231.254:35797)
0619 12:25:04.869653 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:04.869659 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:04.869676 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:25:04.869808 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:04.869813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:04.869858 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:04.869930 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:04.870367 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.870920 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.871904 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.872132 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.872394 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.874966 (+ 2572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:50625: BlockingRecv error: recv got EOF from 127.6.231.195:50625 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:04 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:04.880879 16219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:04.875601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:33157)
0619 12:25:04.875716 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:04.875732 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:04.875815 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:04.876058 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:04.876062 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:04.876253 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:04.876349 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.876356 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.877298 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.877301 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.877932 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:04.877940 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.878407 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.878410 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:04.878474 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.878890 (+ 416us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:04.878902 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:04.880720 (+ 1818us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":30}
W20260619 12:25:04.880967 15954 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:04.875666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:33157 (local address 127.6.231.254:35797)
0619 12:25:04.875800 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:04.875804 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:04.875818 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:25:04.875903 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:04.875907 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:04.875952 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:04.876043 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:04.876519 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.877182 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.878040 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:04.878307 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:04.878390 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:04.880851 (+ 2461us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:33157: BlockingRecv error: recv got EOF from 127.6.231.195:33157 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260619 12:25:04.881156 16217 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Jun 19 12:25:05 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:05 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:05.497067 16222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:05.491541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:52957)
0619 12:25:05.491859 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:05.491872 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:05.491971 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:05.492284 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:05.492286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:05.492463 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:05.492557 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:05.492562 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:05.493359 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:05.493361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:05.493946 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:05.493954 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:05.494445 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:05.494448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:05.494510 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:05.495009 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:05.495021 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:05.496909 (+ 1888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":97,"threads_started":1}
W20260619 12:25:05.497265 16223 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:05.491675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:52957 (local address 127.6.231.254:35797)
0619 12:25:05.491905 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:05.491908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:05.491924 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:25:05.492064 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:05.492066 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:05.492108 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:05.492240 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:05.492678 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:05.493255 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:05.494059 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:05.494342 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:05.494425 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:05.497056 (+ 2631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:52957: BlockingRecv error: recv got EOF from 127.6.231.193:52957 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":55,"threads_started":1}
Jun 19 12:25:05 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:05 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:05.693022 16224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:05.686646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:44559)
0619 12:25:05.686984 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:05.687000 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:05.687131 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:05.687605 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:05.687608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:05.687847 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:05.687971 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:05.687978 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:05.688945 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:05.688952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:05.689579 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:05.689588 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:05.690062 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:05.690065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:05.690145 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:05.690668 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:05.690680 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:05.692897 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":123,"threads_started":1}
W20260619 12:25:05.693218 16223 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:05.686756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:44559 (local address 127.6.231.254:35797)
0619 12:25:05.686892 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:05.686897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:05.687109 (+ 212us) server_negotiation.cc:407] Connection header received
0619 12:25:05.687373 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:05.687375 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:05.687422 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:05.687539 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:05.688170 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:05.688839 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:05.689700 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:05.689932 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:05.690057 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:05.693068 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:44559: BlockingRecv error: recv got EOF from 127.6.231.194:44559 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Jun 19 12:25:05 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:05 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:05.887329 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:05.881824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:60457)
0619 12:25:05.882122 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:05.882137 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:05.882254 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:05.882525 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:05.882528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:05.882733 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:05.882852 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:05.882858 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:05.883692 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:05.883696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:05.884263 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:05.884271 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:05.884723 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:05.884725 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:05.884801 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:05.885375 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:05.885389 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:05.887214 (+ 1825us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":89,"threads_started":1}
W20260619 12:25:05.887497 16223 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:05.881898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:60457 (local address 127.6.231.254:35797)
0619 12:25:05.882055 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:05.882061 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:05.882251 (+ 190us) server_negotiation.cc:407] Connection header received
0619 12:25:05.882328 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:05.882330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:05.882376 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:05.882454 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:05.882994 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:05.883576 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:05.884393 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:05.884612 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:05.884699 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:05.887368 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:60457: BlockingRecv error: recv got EOF from 127.6.231.195:60457 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 12:25:06 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:06 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:06.503528 16226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:06.497934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:49835)
0619 12:25:06.498212 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:06.498226 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:06.498310 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:06.498590 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:06.498592 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:06.498771 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:06.498868 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:06.498873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:06.499624 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:06.499627 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:06.500264 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:06.500273 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:06.500738 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:06.500741 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:06.500810 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:06.501343 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:06.501355 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:06.503378 (+ 2023us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":115,"threads_started":1}
W20260619 12:25:06.503670 16227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:06.498017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:49835 (local address 127.6.231.254:35797)
0619 12:25:06.498260 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:06.498263 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:06.498275 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:25:06.498379 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:06.498381 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:06.498420 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:06.498499 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:06.498976 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:06.499501 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:06.500381 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:06.500606 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:06.500691 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:06.503534 (+ 2843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:49835: BlockingRecv error: recv got EOF from 127.6.231.193:49835 (error 108)
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":97,"threads_started":1}
Jun 19 12:25:06 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:06 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:06.699955 16228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:06.693933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:48859)
0619 12:25:06.694311 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:06.694325 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:06.694439 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:06.694674 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:06.694676 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:06.694858 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:06.694953 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:06.694957 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:06.695751 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:06.695754 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:06.696324 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:06.696332 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:06.696938 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:06.696945 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:06.697027 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:06.697826 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:06.697842 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:06.699816 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":86,"threads_started":1}
W20260619 12:25:06.700232 16227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:06.694194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:48859 (local address 127.6.231.254:35797)
0619 12:25:06.694343 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:06.694346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:06.694387 (+ 41us) server_negotiation.cc:407] Connection header received
0619 12:25:06.694517 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:06.694520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:06.694563 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:06.694635 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:06.695090 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:06.695624 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:06.696493 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:06.696799 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:06.696886 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:06.700097 (+ 3211us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:48859: BlockingRecv error: recv got EOF from 127.6.231.194:48859 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Jun 19 12:25:06 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:06 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:06.894176 16229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:06.888347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:51791)
0619 12:25:06.888616 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:06.888639 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:06.888732 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:06.889135 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:06.889138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:06.889314 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:06.889416 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:06.889421 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:06.890256 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:06.890260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:06.890826 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:06.890834 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:06.891294 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:06.891297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:06.891404 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:06.891964 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:06.891982 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:06.894050 (+ 2068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":81,"threads_started":1}
W20260619 12:25:06.894344 16227 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:06.888511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:51791 (local address 127.6.231.254:35797)
0619 12:25:06.888660 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:06.888664 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:06.888679 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:25:06.888818 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:06.888820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:06.888863 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:06.888939 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:06.889590 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:06.890127 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:06.890951 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:06.891174 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:06.891281 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:06.894212 (+ 2931us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:51791: BlockingRecv error: recv got EOF from 127.6.231.195:51791 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Jun 19 12:25:07 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:07 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:07.510576 16230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:07.504445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:48109)
0619 12:25:07.504746 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:07.504759 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:07.504849 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:07.505157 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:07.505160 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:07.505341 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:07.505523 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:07.505530 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:07.506351 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:07.506354 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:07.507125 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:07.507134 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:07.507720 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:07.507724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:07.507815 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:07.508479 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:07.508492 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:07.510460 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":125,"threads_started":1}
W20260619 12:25:07.510829 16231 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:07.504557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:48109 (local address 127.6.231.254:35797)
0619 12:25:07.504815 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:07.504818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:07.504832 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:25:07.504937 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:07.504939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:07.504987 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:07.505081 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:07.505670 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:07.506238 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:07.507230 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:07.507580 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:07.507727 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:07.510686 (+ 2959us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:48109: BlockingRecv error: recv got EOF from 127.6.231.193:48109 (error 108)
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":110,"threads_started":1}
Jun 19 12:25:07 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:07 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:07.706575 16232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:07.700776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:33331)
0619 12:25:07.701085 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:07.701099 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:07.701194 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:07.701422 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:07.701424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:07.701775 (+ 351us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:07.701914 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:07.701922 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:07.702952 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:07.702959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:07.703532 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:07.703540 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:07.704014 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:07.704018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:07.704094 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:07.704594 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:07.704604 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:07.706394 (+ 1790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":138,"threads_started":1}
W20260619 12:25:07.706620 16231 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:07.700832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:33331 (local address 127.6.231.254:35797)
0619 12:25:07.700986 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:07.700991 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:07.701172 (+ 181us) server_negotiation.cc:407] Connection header received
0619 12:25:07.701273 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:07.701275 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:07.701320 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:07.701400 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:07.702099 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:07.702843 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:07.703658 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:07.703901 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:07.704020 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:07.706518 (+ 2498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:33331: BlockingRecv error: recv got EOF from 127.6.231.194:33331 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 12:25:07 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:07 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:07.904388 16233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:07.895144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:43595)
0619 12:25:07.895436 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:07.895451 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:07.895557 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:07.895812 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:07.895814 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:07.895990 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:07.896080 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:07.896085 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:07.896847 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:07.896849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:07.897409 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:07.897417 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:07.897924 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:07.897927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:07.898002 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:07.902140 (+ 4138us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:07.902154 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:07.904254 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":88,"threads_started":1}
W20260619 12:25:07.904572 16231 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:07.895311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:43595 (local address 127.6.231.254:35797)
0619 12:25:07.895505 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:07.895509 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:07.895523 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:25:07.895649 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:07.895651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:07.895706 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:07.895778 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:07.896193 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:07.896743 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:07.897595 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:07.897820 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:07.897899 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:07.904432 (+ 6533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:43595: BlockingRecv error: recv got EOF from 127.6.231.195:43595 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
Jun 19 12:25:08 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:08 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:08.517863 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:08.511374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:56083)
0619 12:25:08.511675 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:08.511691 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:08.511787 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:08.512096 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:08.512099 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:08.512334 (+ 235us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:08.512482 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:08.512490 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:08.513392 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:08.513396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:08.514115 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:08.514125 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:08.514694 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:08.514697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:08.514764 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:08.515424 (+ 660us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:08.515437 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:08.517732 (+ 2295us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":110,"threads_started":1}
W20260619 12:25:08.518118 16235 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:08.511564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:56083 (local address 127.6.231.254:35797)
0619 12:25:08.511822 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:08.511826 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:08.511839 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:25:08.511879 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:08.511882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:08.511949 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:08.512040 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:08.512621 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:08.513239 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:08.514252 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:08.514568 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:08.514663 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:08.517921 (+ 3258us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:56083: BlockingRecv error: recv got EOF from 127.6.231.193:56083 (error 108)
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":106,"threads_started":1}
Jun 19 12:25:08 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:08 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:08.713330 16236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:08.707453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:52519)
0619 12:25:08.707750 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:08.707768 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:08.707966 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:08.708214 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:08.708216 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:08.708397 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:08.708489 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:08.708496 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:08.709323 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:08.709326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:08.710124 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:08.710133 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:08.710609 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:08.710612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:08.710673 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:08.711267 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:08.711280 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:08.713201 (+ 1921us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":108,"threads_started":1}
W20260619 12:25:08.713521 16235 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:08.707489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:52519 (local address 127.6.231.254:35797)
0619 12:25:08.707696 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:08.707699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:08.707846 (+ 147us) server_negotiation.cc:407] Connection header received
0619 12:25:08.708059 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:08.708062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:08.708106 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:08.708181 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:08.708586 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:08.709215 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:08.710254 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:08.710496 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:08.710577 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:08.713362 (+ 2785us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:52519: BlockingRecv error: recv got EOF from 127.6.231.194:52519 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Jun 19 12:25:08 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:08 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:08.911531 16237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:08.905385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:60923)
0619 12:25:08.905731 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:08.905745 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:08.905918 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:08.906248 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:08.906252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:08.906455 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:08.906550 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:08.906555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:08.907589 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:08.907593 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:08.908281 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:08.908290 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:08.908773 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:08.908776 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:08.908904 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:08.909542 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:08.909554 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:08.911404 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":126,"threads_started":1}
W20260619 12:25:08.911710 16235 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:08.905504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:60923 (local address 127.6.231.254:35797)
0619 12:25:08.905651 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:08.905655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:08.905843 (+ 188us) server_negotiation.cc:407] Connection header received
0619 12:25:08.906052 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:08.906055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:08.906114 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:08.906189 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:08.906694 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:08.907457 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:08.908425 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:08.908658 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:08.908759 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:08.911548 (+ 2789us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:60923: BlockingRecv error: recv got EOF from 127.6.231.195:60923 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Jun 19 12:25:09 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:09 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:09.524730 16238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:09.518766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:48257)
0619 12:25:09.519090 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:09.519109 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:09.519214 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:09.519479 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:09.519481 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:09.519666 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:09.519849 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:09.519856 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:09.520650 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:09.520653 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:09.521394 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:09.521404 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:09.521938 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:09.521941 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:09.522007 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:09.522597 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:09.522609 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:09.524592 (+ 1983us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":120,"threads_started":1}
W20260619 12:25:09.524928 16239 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:09.518965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:48257 (local address 127.6.231.254:35797)
0619 12:25:09.519204 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:09.519207 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:09.519220 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:25:09.519299 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:09.519301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:09.519345 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:09.519417 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:09.519990 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:09.520543 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:09.521558 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:09.521823 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:09.521920 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:09.524775 (+ 2855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:48257: BlockingRecv error: recv got EOF from 127.6.231.193:48257 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":73,"threads_started":1}
Jun 19 12:25:09 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:09 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:09.720459 16240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:09.714457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:60363)
0619 12:25:09.714859 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:09.714875 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:09.714986 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:09.715329 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:09.715332 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:09.715514 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:09.715607 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:09.715612 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:09.716353 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:09.716355 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:09.716958 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:09.716966 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:09.717549 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:09.717552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:09.717655 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:09.718306 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:09.718318 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:09.720276 (+ 1958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":72,"threads_started":1}
W20260619 12:25:09.720499 16239 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:09.714692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:60363 (local address 127.6.231.254:35797)
0619 12:25:09.714866 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:09.714870 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:09.714958 (+ 88us) server_negotiation.cc:407] Connection header received
0619 12:25:09.715130 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:09.715133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:09.715181 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:09.715262 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:09.715722 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:09.716259 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:09.717085 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:09.717356 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:09.717503 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:09.720401 (+ 2898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:60363: BlockingRecv error: recv got EOF from 127.6.231.194:60363 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Jun 19 12:25:09 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:09 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:09.918169 16241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:09.912277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:35997)
0619 12:25:09.912630 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:09.912652 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:09.912782 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:09.913061 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:09.913064 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:09.913293 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:09.913383 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:09.913389 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:09.914265 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:09.914270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:09.914853 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:09.914861 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:09.915337 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:09.915341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:09.915426 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:09.916005 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:09.916016 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:09.917992 (+ 1976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":88,"threads_started":1}
W20260619 12:25:09.918233 16239 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:09.912406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:35997 (local address 127.6.231.254:35797)
0619 12:25:09.912588 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:09.912592 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:09.912741 (+ 149us) server_negotiation.cc:407] Connection header received
0619 12:25:09.912895 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:09.912899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:09.912952 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:09.913040 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:09.913557 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:09.914144 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:09.914988 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:09.915220 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:09.915337 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:09.918119 (+ 2782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:35997: BlockingRecv error: recv got EOF from 127.6.231.195:35997 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Jun 19 12:25:10 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:10 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:10.531528 16242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:10.525632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:56655)
0619 12:25:10.525944 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:10.525959 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:10.526082 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:10.526415 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:10.526417 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:10.526600 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:10.526693 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:10.526698 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:10.527515 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:10.527517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:10.528082 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:10.528090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:10.528521 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:10.528524 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:10.528590 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:10.529216 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:10.529229 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:10.531407 (+ 2178us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":109,"threads_started":1}
W20260619 12:25:10.531785 16243 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:10.525708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:56655 (local address 127.6.231.254:35797)
0619 12:25:10.525976 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:10.525980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:10.526011 (+ 31us) server_negotiation.cc:407] Connection header received
0619 12:25:10.526225 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:10.526228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:10.526297 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:10.526378 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:10.526866 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:10.527418 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:10.528183 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:10.528416 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:10.528492 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:10.531601 (+ 3109us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:56655: BlockingRecv error: recv got EOF from 127.6.231.193:56655 (error 108)
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":66,"threads_started":1}
Jun 19 12:25:10 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:10 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:10.727523 16244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:10.721118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:37701)
0619 12:25:10.721419 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:10.721450 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:10.721566 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:10.721865 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:10.721867 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:10.722062 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:10.722160 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:10.722166 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:10.723360 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:10.723364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:10.724139 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:10.724151 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:10.724912 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:10.724918 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:10.724997 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:10.725591 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:10.725603 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:10.727404 (+ 1801us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":87,"threads_started":1}
W20260619 12:25:10.727792 16243 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:10.721235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:37701 (local address 127.6.231.254:35797)
0619 12:25:10.721414 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:10.721421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:10.721531 (+ 110us) server_negotiation.cc:407] Connection header received
0619 12:25:10.721658 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:10.721662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:10.721729 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:10.721824 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:10.722271 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:10.723249 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:10.724286 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:10.724778 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:10.724927 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:10.727624 (+ 2697us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:37701: BlockingRecv error: recv got EOF from 127.6.231.194:37701 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Jun 19 12:25:10 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:10 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:10.925024 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:10.919042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:39347)
0619 12:25:10.919356 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:10.919378 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:10.919502 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:10.919871 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:10.919874 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:10.920094 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:10.920229 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:10.920235 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:10.921005 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:10.921011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:10.921672 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:10.921681 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:10.922167 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:10.922171 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:10.922274 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:10.922813 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:10.922826 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:10.924897 (+ 2071us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":77,"threads_started":1}
W20260619 12:25:10.925231 16243 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:10.919278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:39347 (local address 127.6.231.254:35797)
0619 12:25:10.919427 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:10.919432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:10.919448 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:25:10.919624 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:10.919628 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:10.919750 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:10.919847 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:10.920354 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:10.920896 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:10.921789 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:10.922017 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:10.922133 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:10.925027 (+ 2894us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:39347: BlockingRecv error: recv got EOF from 127.6.231.195:39347 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Jun 19 12:25:11 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:11 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:11.538633 16246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:11.532508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:51707)
0619 12:25:11.532839 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:11.532854 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:11.532950 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:11.533224 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:11.533226 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:11.533417 (+ 191us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:11.533547 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:11.533553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:11.534367 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:11.534373 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:11.534938 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:11.534946 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:11.535438 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:11.535441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:11.535536 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:11.536084 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:11.536095 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:11.538453 (+ 2358us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":98,"threads_started":1}
W20260619 12:25:11.538708 16247 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:11.532724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:51707 (local address 127.6.231.254:35797)
0619 12:25:11.532991 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:11.532994 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:11.533008 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:25:11.533050 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:11.533052 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:11.533093 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:11.533183 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:11.533670 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:11.534243 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:11.535067 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:11.535309 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:11.535419 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:11.538579 (+ 3160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:51707: BlockingRecv error: recv got EOF from 127.6.231.193:51707 (error 108)
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":108,"threads_started":1}
Jun 19 12:25:11 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:11 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:11.734396 16248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:11.728350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:33859)
0619 12:25:11.728705 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:11.728719 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:11.728816 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:11.729094 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:11.729097 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:11.729317 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:11.729416 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:11.729422 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:11.730319 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:11.730322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:11.731047 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:11.731057 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:11.731575 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:11.731578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:11.731642 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:11.732330 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:11.732342 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:11.734220 (+ 1878us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":106,"threads_started":1}
W20260619 12:25:11.734488 16247 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:11.728391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:33859 (local address 127.6.231.254:35797)
0619 12:25:11.728537 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:11.728541 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:11.728820 (+ 279us) server_negotiation.cc:407] Connection header received
0619 12:25:11.728896 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:11.728899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:11.728948 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:11.729017 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:11.729588 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:11.730200 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:11.731180 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:11.731449 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:11.731537 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:11.734367 (+ 2830us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:33859: BlockingRecv error: recv got EOF from 127.6.231.194:33859 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 12:25:11 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:11 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:11.931834 16249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:11.925812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:45643)
0619 12:25:11.926143 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:11.926156 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:11.926261 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:11.926498 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:11.926501 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:11.926745 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:11.926836 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:11.926842 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:11.927610 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:11.927616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:11.928184 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:11.928191 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:11.928656 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:11.928659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:11.928753 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:11.929334 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:11.929346 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:11.931659 (+ 2313us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":113,"threads_started":1}
W20260619 12:25:11.931869 16247 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:11.925883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:45643 (local address 127.6.231.254:35797)
0619 12:25:11.926033 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:11.926037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:11.926233 (+ 196us) server_negotiation.cc:407] Connection header received
0619 12:25:11.926334 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:11.926336 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:11.926381 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:11.926478 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:11.926961 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:11.927501 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:11.928307 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:11.928540 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:11.928652 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:11.931768 (+ 3116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:45643: BlockingRecv error: recv got EOF from 127.6.231.195:45643 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Jun 19 12:25:12 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:12 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:12.544979 16250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:12.539364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:43757)
0619 12:25:12.539746 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:12.539762 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:12.539860 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:12.540110 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:12.540112 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:12.540284 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:12.540376 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:12.540381 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:12.541205 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:12.541208 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:12.541781 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:12.541789 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:12.542196 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:12.542198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:12.542256 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:12.542759 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:12.542771 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:12.544806 (+ 2035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
W20260619 12:25:12.545074 16251 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:12.539567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:43757 (local address 127.6.231.254:35797)
0619 12:25:12.539808 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:12.539811 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:12.539825 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:25:12.539944 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:12.539946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:12.539989 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:12.540063 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:12.540479 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:12.541110 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:12.541873 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:12.542105 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:12.542180 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:12.544937 (+ 2757us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:43757: BlockingRecv error: recv got EOF from 127.6.231.193:43757 (error 108)
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":92,"threads_started":1}
Jun 19 12:25:12 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:12 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:12.741741 16252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:12.735320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:59757)
0619 12:25:12.735643 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:12.735657 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:12.735762 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:12.736040 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:12.736043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:12.736240 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:12.736340 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:12.736346 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:12.737316 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:12.737319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:12.738034 (+ 715us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:12.738043 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:12.738488 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:12.738491 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:12.738610 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:12.739174 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:12.739186 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:12.741564 (+ 2378us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":83,"threads_started":1}
W20260619 12:25:12.741904 16251 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:12.735377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:59757 (local address 127.6.231.254:35797)
0619 12:25:12.735515 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:12.735520 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:12.735731 (+ 211us) server_negotiation.cc:407] Connection header received
0619 12:25:12.735870 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:12.735873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:12.735919 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:12.735993 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:12.736446 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:12.737193 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:12.738139 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:12.738378 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:12.738476 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:12.741795 (+ 3319us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:59757: BlockingRecv error: recv got EOF from 127.6.231.194:59757 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 12:25:12 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:12 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:12.943718 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:12.932691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:52065)
0619 12:25:12.932995 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:12.933009 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:12.933113 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:12.933355 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:12.933358 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:12.933605 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:12.933741 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:12.933747 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:12.934723 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:12.934729 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:12.935615 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:12.935625 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:12.936045 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:12.936049 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:12.936114 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:12.941549 (+ 5435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:12.941582 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:12.943590 (+ 2008us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":88,"threads_started":1}
W20260619 12:25:12.943922 16251 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:12.932773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:52065 (local address 127.6.231.254:35797)
0619 12:25:12.932907 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:12.932911 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:12.933084 (+ 173us) server_negotiation.cc:407] Connection header received
0619 12:25:12.933203 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:12.933206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:12.933252 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:12.933315 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:12.933885 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:12.934587 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:12.935728 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:12.935942 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:12.936029 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:12.943759 (+ 7730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:52065: BlockingRecv error: recv got EOF from 127.6.231.195:52065 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Jun 19 12:25:13 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:13 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:13.551612 16254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:13.545853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:51835)
0619 12:25:13.546176 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:13.546191 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:13.546282 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:13.546535 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:13.546537 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:13.546720 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:13.546814 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:13.546819 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:13.547643 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:13.547645 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:13.548217 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:13.548225 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:13.548709 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:13.548715 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:13.548807 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:13.549519 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:13.549536 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:13.551478 (+ 1942us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":90,"threads_started":1}
W20260619 12:25:13.551788 16255 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:13.545987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:51835 (local address 127.6.231.254:35797)
0619 12:25:13.546229 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:13.546233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:13.546246 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:25:13.546377 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:13.546379 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:13.546422 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:13.546500 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:13.546926 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:13.547493 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:13.548325 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:13.548563 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:13.548645 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:13.551616 (+ 2971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:51835: BlockingRecv error: recv got EOF from 127.6.231.193:51835 (error 108)
Metrics: {"server-negotiator.queue_time_us":156,"thread_start_us":61,"threads_started":1}
Jun 19 12:25:13 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:13 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:13.748365 16256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:13.742644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:56839)
0619 12:25:13.742942 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:13.742960 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:13.743105 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:13.743396 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:13.743399 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:13.743594 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:13.743694 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:13.743700 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:13.744486 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:13.744489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:13.745069 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:13.745077 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:13.745523 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:13.745525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:13.745617 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:13.746114 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:13.746126 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:13.748229 (+ 2103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":85,"threads_started":1}
W20260619 12:25:13.748497 16255 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:13.742709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:56839 (local address 127.6.231.254:35797)
0619 12:25:13.742964 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:13.742968 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:13.743062 (+ 94us) server_negotiation.cc:407] Connection header received
0619 12:25:13.743198 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:13.743201 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:13.743260 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:13.743339 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:13.743804 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:13.744386 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:13.745180 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:13.745404 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:13.745512 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:13.748388 (+ 2876us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:56839: BlockingRecv error: recv got EOF from 127.6.231.194:56839 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 12:25:13 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:13 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:13.950150 16257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:13.944738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:52149)
0619 12:25:13.944979 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:13.945001 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:13.945103 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:13.945333 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:13.945335 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:13.945573 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:13.945671 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:13.945676 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:13.946398 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:13.946400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:13.946996 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:13.947005 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:13.947446 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:13.947448 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:13.947539 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:13.948105 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:13.948117 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:13.950027 (+ 1910us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":89,"threads_started":1}
W20260619 12:25:13.950313 16255 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:13.944769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:52149 (local address 127.6.231.254:35797)
0619 12:25:13.944900 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:13.944904 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:13.945075 (+ 171us) server_negotiation.cc:407] Connection header received
0619 12:25:13.945185 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:13.945187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:13.945232 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:13.945301 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:13.945775 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:13.946310 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:13.947105 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:13.947334 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:13.947413 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:13.950179 (+ 2766us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:52149: BlockingRecv error: recv got EOF from 127.6.231.195:52149 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 12:25:14 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:14 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:14.558589 16258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:14.552492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:60817)
0619 12:25:14.552790 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:14.552804 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:14.552893 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:14.553152 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:14.553155 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:14.553342 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:14.553501 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:14.553508 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:14.554307 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:14.554367 (+ 60us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:14.555003 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:14.555013 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:14.555519 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:14.555523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:14.555619 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:14.556259 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:14.556274 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:14.558469 (+ 2195us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":71,"threads_started":1}
W20260619 12:25:14.558806 16259 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:14.552552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:60817 (local address 127.6.231.254:35797)
0619 12:25:14.552836 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:14.552839 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:14.552852 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:25:14.552986 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:14.552988 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:14.553029 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:14.553107 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:14.553608 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:14.554178 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:14.555136 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:14.555361 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:14.555497 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:14.558623 (+ 3126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:60817: BlockingRecv error: recv got EOF from 127.6.231.193:60817 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":110,"threads_started":1}
Jun 19 12:25:14 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:14 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:14.754649 16260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:14.749255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:57863)
0619 12:25:14.749545 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:14.749559 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:14.749642 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:14.749890 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:14.749892 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:14.750067 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:14.750162 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:14.750167 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:14.750876 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:14.750879 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:14.751431 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:14.751438 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:14.751898 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:14.751901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:14.751978 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:14.752623 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:14.752635 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:14.754530 (+ 1895us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":67,"threads_started":1}
W20260619 12:25:14.754797 16259 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:14.749341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:57863 (local address 127.6.231.254:35797)
0619 12:25:14.749582 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:14.749585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:14.749601 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:25:14.749726 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:14.749728 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:14.749769 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:14.749835 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:14.750256 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:14.750765 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:14.751531 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:14.751750 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:14.751827 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:14.754676 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:57863: BlockingRecv error: recv got EOF from 127.6.231.194:57863 (error 108)
Metrics: {"server-negotiator.queue_time_us":162}
Jun 19 12:25:14 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:14 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:14.956774 16261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:14.950998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:42979)
0619 12:25:14.951346 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:14.951362 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:14.951511 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:14.951789 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:14.951793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:14.951996 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:14.952089 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:14.952094 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:14.952897 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:14.952900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:14.953494 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:14.953502 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:14.953998 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:14.954002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:14.954084 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:14.954578 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:14.954590 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:14.956653 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":114,"threads_started":1}
W20260619 12:25:14.956861 16259 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:14.951189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:42979 (local address 127.6.231.254:35797)
0619 12:25:14.951347 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:14.951353 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:14.951451 (+ 98us) server_negotiation.cc:407] Connection header received
0619 12:25:14.951614 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:14.951619 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:14.951671 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:14.951762 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:14.952209 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:14.952755 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:14.953650 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:14.953884 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:14.953994 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:14.956771 (+ 2777us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:42979: BlockingRecv error: recv got EOF from 127.6.231.195:42979 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
Jun 19 12:25:15 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:15 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:15.564925 16262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:15.559349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:57225)
0619 12:25:15.559649 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:15.559664 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:15.559757 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:15.560008 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:15.560009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:15.560187 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:15.560281 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:15.560287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:15.561164 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:15.561167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:15.561746 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:15.561753 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:15.562195 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:15.562198 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:15.562258 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:15.562766 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:15.562778 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:15.564739 (+ 1961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":86,"threads_started":1}
W20260619 12:25:15.564988 16263 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:15.559486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:57225 (local address 127.6.231.254:35797)
0619 12:25:15.559707 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:15.559710 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:15.559724 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:25:15.559871 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:15.559874 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:15.559917 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:15.559995 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:15.560431 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:15.561026 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:15.561860 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:15.562090 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:15.562178 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:15.564868 (+ 2690us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:57225: BlockingRecv error: recv got EOF from 127.6.231.193:57225 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":61,"threads_started":1}
Jun 19 12:25:15 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:15 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:15.761488 16264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:15.755507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:54899)
0619 12:25:15.755868 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:15.755894 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:15.755993 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:15.756369 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:15.756373 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:15.756623 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:15.756756 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:15.756767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:15.757603 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:15.757608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:15.758183 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:15.758191 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:15.758678 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:15.758682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:15.758762 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:15.759334 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:15.759346 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:15.761276 (+ 1930us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":120,"threads_started":1}
W20260619 12:25:15.761545 16263 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:15.755667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:54899 (local address 127.6.231.254:35797)
0619 12:25:15.755839 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:15.755865 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:15.755967 (+ 102us) server_negotiation.cc:407] Connection header received
0619 12:25:15.756114 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:15.756117 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:15.756172 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:15.756275 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:15.756864 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:15.757482 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:15.758312 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:15.758561 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:15.758677 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:15.761402 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:54899: BlockingRecv error: recv got EOF from 127.6.231.194:54899 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Jun 19 12:25:15 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:15 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:15.963251 16265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:15.957771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:58285)
0619 12:25:15.958053 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:15.958066 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:15.958195 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:15.958522 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:15.958525 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:15.958732 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:15.958844 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:15.958849 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:15.959616 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:15.959622 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:15.960212 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:15.960219 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:15.960672 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:15.960675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:15.960760 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:15.961277 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:15.961289 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:15.963084 (+ 1795us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":85,"threads_started":1}
W20260619 12:25:15.963321 16263 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:15.957797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:58285 (local address 127.6.231.254:35797)
0619 12:25:15.957959 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:15.957964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:15.958141 (+ 177us) server_negotiation.cc:407] Connection header received
0619 12:25:15.958349 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:15.958351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:15.958396 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:15.958501 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:15.958966 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:15.959509 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:15.960334 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:15.960560 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:15.960668 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:15.963203 (+ 2535us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:58285: BlockingRecv error: recv got EOF from 127.6.231.195:58285 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Jun 19 12:25:16 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:16 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:16.571094 16266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:16.565769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:53743)
0619 12:25:16.566127 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:16.566141 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:16.566241 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:16.566482 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:16.566484 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:16.566660 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:16.566775 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:16.566780 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:16.567496 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:16.567498 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:16.568079 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:16.568086 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:16.568494 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:16.568497 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:16.568559 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:16.569082 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:16.569093 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:16.570978 (+ 1885us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":192,"threads_started":1}
W20260619 12:25:16.571252 16267 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:16.565879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:53743 (local address 127.6.231.254:35797)
0619 12:25:16.566119 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:16.566123 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:16.566210 (+ 87us) server_negotiation.cc:407] Connection header received
0619 12:25:16.566321 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:16.566325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:16.566368 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:16.566448 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:16.566881 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:16.567406 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:16.568175 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:16.568399 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:16.568473 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:16.571119 (+ 2646us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:53743: BlockingRecv error: recv got EOF from 127.6.231.193:53743 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":92,"threads_started":1}
Jun 19 12:25:16 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:16 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:16.768087 16268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:16.762276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:35347)
0619 12:25:16.762542 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:16.762555 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:16.762649 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:16.762939 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:16.762941 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:16.763123 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:16.763227 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:16.763232 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:16.764101 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:16.764104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:16.764704 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:16.764713 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:16.765156 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:16.765158 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:16.765252 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:16.765789 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:16.765801 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:16.767941 (+ 2140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":102,"threads_started":1}
W20260619 12:25:16.768239 16267 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:16.762351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:35347 (local address 127.6.231.254:35797)
0619 12:25:16.762498 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:16.762501 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:16.762621 (+ 120us) server_negotiation.cc:407] Connection header received
0619 12:25:16.762758 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:16.762760 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:16.762803 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:16.762873 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:16.763378 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:16.763971 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:16.764832 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:16.765050 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:16.765137 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:16.768097 (+ 2960us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:35347: BlockingRecv error: recv got EOF from 127.6.231.194:35347 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Jun 19 12:25:16 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:16 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:16.971300 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:16.964110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:39105)
0619 12:25:16.964420 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:16.964433 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:16.964560 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:16.964847 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:16.964850 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:16.965065 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:16.965169 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:16.965177 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:16.966119 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:16.966125 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:16.966980 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:16.966990 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:16.967643 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:16.967650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:16.967776 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:16.968513 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:16.968534 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:16.971105 (+ 2571us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":81,"threads_started":1}
W20260619 12:25:16.971362 16267 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:16.964216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:39105 (local address 127.6.231.254:35797)
0619 12:25:16.964353 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:16.964358 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:16.964505 (+ 147us) server_negotiation.cc:407] Connection header received
0619 12:25:16.964675 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:16.964679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:16.964735 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:16.964826 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:16.965304 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:16.966022 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:16.967148 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:16.967486 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:16.967604 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:16.971247 (+ 3643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:39105: BlockingRecv error: recv got EOF from 127.6.231.195:39105 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Jun 19 12:25:17 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:17 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:17.578238 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:17.572001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:49405)
0619 12:25:17.572349 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:17.572363 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:17.572453 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:17.572725 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:17.572727 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:17.572909 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:17.573002 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:17.573008 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:17.573892 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:17.573894 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:17.574479 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:17.574488 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:17.575037 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:17.575040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:17.575109 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:17.575794 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:17.575807 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:17.578025 (+ 2218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":173,"threads_started":1}
W20260619 12:25:17.578301 16271 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:17.572189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:49405 (local address 127.6.231.254:35797)
0619 12:25:17.572445 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:17.572449 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:17.572463 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:25:17.572533 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:17.572535 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:17.572580 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:17.572687 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:17.573124 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:17.573778 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:17.574639 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:17.574896 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:17.574990 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:17.578163 (+ 3173us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:49405: BlockingRecv error: recv got EOF from 127.6.231.193:49405 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":98,"threads_started":1}
Jun 19 12:25:17 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:17 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:17.774346 16272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:17.769005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:56525)
0619 12:25:17.769292 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:17.769305 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:17.769386 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:17.769671 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:17.769673 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:17.769852 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:17.769946 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:17.769952 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:17.770780 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:17.770782 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:17.771351 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:17.771359 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:17.771823 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:17.771826 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:17.771891 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:17.772366 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:17.772377 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:17.774224 (+ 1847us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":98,"threads_started":1}
W20260619 12:25:17.774479 16271 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:17.769170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:56525 (local address 127.6.231.254:35797)
0619 12:25:17.769342 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:17.769346 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:17.769363 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:25:17.769483 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:17.769486 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:17.769533 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:17.769656 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:17.770057 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:17.770672 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:17.771483 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:17.771715 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:17.771801 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:17.774364 (+ 2563us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:56525: BlockingRecv error: recv got EOF from 127.6.231.194:56525 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Jun 19 12:25:17 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:17 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:17.983600 16273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:17.972179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:34613)
0619 12:25:17.972461 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:17.972474 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:17.972576 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:17.972806 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:17.972808 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:17.972988 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:17.973076 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:17.973081 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:17.973943 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:17.973946 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:17.974507 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:17.974515 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:17.974992 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:17.974995 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:17.975055 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:17.981508 (+ 6453us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:17.981522 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:17.983469 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":110,"threads_started":1}
W20260619 12:25:17.983835 16271 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:17.972285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:34613 (local address 127.6.231.254:35797)
0619 12:25:17.972423 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:17.972426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:17.972582 (+ 156us) server_negotiation.cc:407] Connection header received
0619 12:25:17.972641 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:17.972643 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:17.972691 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:17.972775 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:17.973196 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:17.973843 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:17.974645 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:17.974871 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:17.974962 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:17.983714 (+ 8752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:34613: BlockingRecv error: recv got EOF from 127.6.231.195:34613 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Jun 19 12:25:18 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:18 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:18.585206 16274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:18.579150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:58745)
0619 12:25:18.579522 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:18.579537 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:18.579636 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:18.579883 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:18.579885 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:18.580073 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:18.580167 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:18.580171 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:18.581121 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:18.581125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:18.581799 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:18.581809 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:18.582355 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:18.582359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:18.582455 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:18.583016 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:18.583038 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:18.585032 (+ 1994us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":102,"threads_started":1}
W20260619 12:25:18.585381 16275 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:18.579312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:58745 (local address 127.6.231.254:35797)
0619 12:25:18.579630 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:18.579634 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:18.579649 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:25:18.579714 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:18.579716 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:18.579763 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:18.579847 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:18.580288 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:18.580989 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:18.581940 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:18.582223 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:18.582349 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:18.585250 (+ 2901us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:58745: BlockingRecv error: recv got EOF from 127.6.231.193:58745 (error 108)
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":72,"threads_started":1}
Jun 19 12:25:18 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:18 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:18.780818 16276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:18.775234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:47289)
0619 12:25:18.775506 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:18.775520 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:18.775615 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:18.775881 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:18.775883 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:18.776062 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:18.776155 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:18.776161 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:18.776902 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:18.776904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:18.777516 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:18.777526 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:18.777958 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:18.777970 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:18.778094 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:18.778585 (+ 491us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:18.778596 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:18.780684 (+ 2088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":99,"threads_started":1}
W20260619 12:25:18.781037 16275 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:18.775441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:47289 (local address 127.6.231.254:35797)
0619 12:25:18.775585 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:18.775589 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:18.775602 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:25:18.775725 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:18.775727 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:18.775772 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:18.775846 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:18.776263 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:18.776801 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:18.777613 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:18.777831 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:18.777908 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:18.780841 (+ 2933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:47289: BlockingRecv error: recv got EOF from 127.6.231.194:47289 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Jun 19 12:25:18 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:18 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.195@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:18.990518 16277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:18.984468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.195:43295)
0619 12:25:18.984825 (+ 357us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:18.984838 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:18.984973 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:18.985304 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:18.985307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:18.985588 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:18.985719 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:18.985726 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:18.986609 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:18.986612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:18.987180 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:18.987188 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:18.987672 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:18.987675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:18.987746 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:18.988321 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:18.988333 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:18.990370 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":82,"threads_started":1}
W20260619 12:25:18.990635 16275 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:18.984590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:43295 (local address 127.6.231.254:35797)
0619 12:25:18.984722 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:18.984726 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:18.984924 (+ 198us) server_negotiation.cc:407] Connection header received
0619 12:25:18.985098 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:18.985101 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:18.985157 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:18.985253 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:18.985873 (+ 620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:18.986493 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:18.987290 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:18.987528 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:18.987611 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:18.990509 (+ 2898us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.195:43295: BlockingRecv error: recv got EOF from 127.6.231.195:43295 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Jun 19 12:25:19 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:19 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.193@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:19.592207 16278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:19.586235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.193:36331)
0619 12:25:19.586550 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:19.586565 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:19.586652 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:19.586930 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:19.586932 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:19.587156 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:19.587253 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:19.587259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:19.588124 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:19.588127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:19.588699 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:19.588707 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:19.589196 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:19.589199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:19.589260 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:19.589940 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:19.589952 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:19.592077 (+ 2125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":79,"threads_started":1}
W20260619 12:25:19.592374 16279 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:19.586306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:36331 (local address 127.6.231.254:35797)
0619 12:25:19.586618 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:19.586621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:19.586635 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:25:19.586737 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:19.586739 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:19.586780 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:19.586859 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:19.587392 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:19.587984 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:19.588846 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:19.589088 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:19.589177 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:19.592225 (+ 3048us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.193:36331: BlockingRecv error: recv got EOF from 127.6.231.193:36331 (error 108)
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":145,"threads_started":1}
Jun 19 12:25:19 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
Jun 19 12:25:19 dist-test-slave-kr9h krb5kdc[15221](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.6.231.194@KRBTEST.COM for oryx/127.6.231.254@KRBTEST.COM, Server not found in Kerberos database
I20260619 12:25:19.787976 16280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:19.781852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35797 (local address 127.6.231.194:53641)
0619 12:25:19.782137 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:19.782151 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:19.782251 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:19.782464 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:19.782466 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:19.782642 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:19.782730 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:19.782735 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:19.783573 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:19.783576 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:19.784177 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:19.784187 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:19.784683 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:19.784686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:19.784756 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:19.785321 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:19.785337 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:19.787849 (+ 2512us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.6.231.254:35797: Server oryx/127.6.231.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":95,"threads_started":1}
W20260619 12:25:19.788153 16279 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:19.781936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:53641 (local address 127.6.231.254:35797)
0619 12:25:19.782087 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:19.782090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:19.782258 (+ 168us) server_negotiation.cc:407] Connection header received
0619 12:25:19.782319 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:19.782321 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:19.782370 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:19.782453 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:19.782836 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:19.783435 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:19.784309 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:19.784544 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:19.784649 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:19.788000 (+ 3351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.6.231.194:53641: BlockingRecv error: recv got EOF from 127.6.231.194:53641 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 12:25:19.869381 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 15804
I20260619 12:25:19.875080 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 15956
I20260619 12:25:19.880151 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 16088
I20260619 12:25:19.885552 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 15734
2026-06-19T12:25:19Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18017 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-06-19T12:25:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:25:19Z Disabled control of system clock
I20260619 12:25:19.916409 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:44801
--webserver_interface=127.6.231.254
--webserver_port=0
--builtin_ntp_servers=127.6.231.212:43077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:44801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260619 12:25:20.029259 16284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:20.029587 16284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:20.029623 16284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:20.033764 16284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:25:20.033824 16284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:20.033840 16284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:20.033859 16284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:25:20.033880 16284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:25:20.038575 16284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:43077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:44801
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:44801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16284
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:20.039640 16284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:20.040969 16284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:20.049778 16292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:20.049935 16284 server_base.cc:1034] running on GCE node
W20260619 12:25:20.049808 16290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:20.049901 16289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:20.050658 16284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:20.051815 16284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:20.052984 16284 hybrid_clock.cc:648] HybridClock initialized: now 1781871920052970 us; error 27 us; skew 500 ppm
I20260619 12:25:20.054867 16284 webserver.cc:460] Webserver started at http://127.6.231.254:46267/ using document root <none> and password file <none>
I20260619 12:25:20.055406 16284 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:20.055466 16284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:20.055630 16284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:20.057667 16284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "ec99920a53a04758a1b09acc4d2f5ed6"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "6914a1f25b0542afbfc76da228ddfc71"
server_key_iv: "0984835fe507dfa868b89f9a994297a1"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.058171 16284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "ec99920a53a04758a1b09acc4d2f5ed6"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "6914a1f25b0542afbfc76da228ddfc71"
server_key_iv: "0984835fe507dfa868b89f9a994297a1"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.061723 16284 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:25:20.064040 16298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:20.065265 16284 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:25:20.065388 16284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "ec99920a53a04758a1b09acc4d2f5ed6"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "6914a1f25b0542afbfc76da228ddfc71"
server_key_iv: "0984835fe507dfa868b89f9a994297a1"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.065521 16284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:20.079777 16284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:20.080641 16284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:20.080813 16284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:20.087827 16284 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:44801
I20260619 12:25:20.087838 16350 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:44801 every 8 connection(s)
I20260619 12:25:20.088872 16284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:25:20.092165 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 16284
I20260619 12:25:20.092288 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:25:20.092377 16351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:25:20.092682 7071 external_mini_cluster.cc:1428] Setting key 433e8bd8712f688595ed478802f7d65b
I20260619 12:25:20.099503 16351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6: Bootstrap starting.
I20260619 12:25:20.101812 16351 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6: Neither blocks nor log segments found. Creating new log.
I20260619 12:25:20.102435 16354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:20.094303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49912 (local address 127.6.231.254:44801)
0619 12:25:20.095068 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:20.095087 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:20.095134 (+ 47us) server_negotiation.cc:407] Connection header received
0619 12:25:20.095903 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:20.095923 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:20.096310 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:20.096593 (+ 283us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:20.096916 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.097955 (+ 1039us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.099133 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.099582 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.099752 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:20.100694 (+ 942us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:20.100699 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:20.100711 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:25:20.101120 (+ 409us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:20.101173 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:25:20.101191 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:20.101301 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:20.101412 (+ 111us) server_negotiation.cc:299] Negotiation successful
0619 12:25:20.101696 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":148,"threads_started":1}
I20260619 12:25:20.102582 16351 log.cc:824] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6: Log is configured to *not* fsync() on all Append() calls
I20260619 12:25:20.104573 16351 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6: No bootstrap required, opened a new log
I20260619 12:25:20.108026 16351 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec99920a53a04758a1b09acc4d2f5ed6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 44801 } }
I20260619 12:25:20.108278 16351 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:25:20.108351 16351 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec99920a53a04758a1b09acc4d2f5ed6, State: Initialized, Role: FOLLOWER
I20260619 12:25:20.108812 16351 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec99920a53a04758a1b09acc4d2f5ed6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 44801 } }
I20260619 12:25:20.108968 16351 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:25:20.109048 16351 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:25:20.109148 16351 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:20.110114 16351 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec99920a53a04758a1b09acc4d2f5ed6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 44801 } }
I20260619 12:25:20.110460 16351 leader_election.cc:302] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec99920a53a04758a1b09acc4d2f5ed6; no voters:
I20260619 12:25:20.110761 16351 leader_election.cc:288] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:25:20.110981 16356 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:25:20.111220 16356 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [term 1 LEADER]: Becoming Leader. State: Replica: ec99920a53a04758a1b09acc4d2f5ed6, State: Running, Role: LEADER
I20260619 12:25:20.111528 16356 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec99920a53a04758a1b09acc4d2f5ed6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 44801 } }
I20260619 12:25:20.112028 16351 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:25:20.113168 16358 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec99920a53a04758a1b09acc4d2f5ed6. Latest consensus state: current_term: 1 leader_uuid: "ec99920a53a04758a1b09acc4d2f5ed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec99920a53a04758a1b09acc4d2f5ed6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 44801 } } }
I20260619 12:25:20.113300 16358 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [sys.catalog]: This master's current role is: LEADER
I20260619 12:25:20.113556 16357 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec99920a53a04758a1b09acc4d2f5ed6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec99920a53a04758a1b09acc4d2f5ed6" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 44801 } } }
I20260619 12:25:20.113734 16365 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:25:20.113737 16357 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6 [sys.catalog]: This master's current role is: LEADER
I20260619 12:25:20.117022 16365 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:25:20.122458 16365 catalog_manager.cc:1347] Generated new cluster ID: ada95c4f87b54483b8b1cbf38c70d134
I20260619 12:25:20.122527 16365 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:25:20.142179 16365 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:25:20.143139 16365 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:25:20.151250 16365 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec99920a53a04758a1b09acc4d2f5ed6: Generated new TSK 0
I20260619 12:25:20.151966 16365 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:25:20.161262 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:44801
--builtin_ntp_servers=127.6.231.212:43077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260619 12:25:20.273883 16375 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:20.274166 16375 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:20.274204 16375 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:20.278365 16375 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:20.278425 16375 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:20.278487 16375 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:25:20.283342 16375 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:43077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:44801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16375
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:20.284484 16375 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:20.285933 16375 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:20.295552 16381 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:20.295569 16380 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:20.295593 16383 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:20.295922 16375 server_base.cc:1034] running on GCE node
I20260619 12:25:20.296362 16375 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:20.297019 16375 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:20.298218 16375 hybrid_clock.cc:648] HybridClock initialized: now 1781871920298204 us; error 26 us; skew 500 ppm
I20260619 12:25:20.300204 16375 webserver.cc:460] Webserver started at http://127.6.231.193:34025/ using document root <none> and password file <none>
I20260619 12:25:20.300784 16375 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:20.300886 16375 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:20.301118 16375 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:20.303145 16375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "799f1215ca4442e1b539487bbd8b6209"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "dd1b3979ccae413d4c8d0f83802fe219"
server_key_iv: "c347c62436db40450ba050dc9ff1c2dc"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.303685 16375 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "799f1215ca4442e1b539487bbd8b6209"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "dd1b3979ccae413d4c8d0f83802fe219"
server_key_iv: "c347c62436db40450ba050dc9ff1c2dc"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.307442 16375 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 12:25:20.310011 16389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:20.311280 16375 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:25:20.311445 16375 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "799f1215ca4442e1b539487bbd8b6209"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "dd1b3979ccae413d4c8d0f83802fe219"
server_key_iv: "c347c62436db40450ba050dc9ff1c2dc"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.311569 16375 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:20.333817 16375 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:20.334568 16375 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:20.334775 16375 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:20.335397 16375 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:20.336334 16375 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:20.336407 16375 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:20.336479 16375 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:20.336515 16375 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:20.345252 16375 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:38419
I20260619 12:25:20.345273 16502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:38419 every 8 connection(s)
I20260619 12:25:20.346184 16375 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:25:20.347965 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 16375
I20260619 12:25:20.348100 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:25:20.348372 7071 external_mini_cluster.cc:1428] Setting key f7311353e6846b1766a725a9aa05c833
I20260619 12:25:20.350555 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:44801
--builtin_ntp_servers=127.6.231.212:43077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 12:25:20.356276 16354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:20.348116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:39131 (local address 127.6.231.254:44801)
0619 12:25:20.348842 (+ 726us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:20.348847 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:20.349496 (+ 649us) server_negotiation.cc:407] Connection header received
0619 12:25:20.351319 (+ 1823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:20.351340 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:20.351385 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:20.351473 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:20.352549 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.353085 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.354362 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.354589 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.354701 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:20.355782 (+ 1081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:20.355785 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:20.355788 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:25:20.355807 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:20.355826 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:25:20.355834 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:20.355912 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:20.356071 (+ 159us) server_negotiation.cc:299] Negotiation successful
0619 12:25:20.356135 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603}
I20260619 12:25:20.357115 16505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:20.348529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:44801 (local address 127.6.231.193:39131)
0619 12:25:20.349295 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:20.349349 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:20.351053 (+ 1704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:20.351673 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:20.351685 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:20.351802 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:20.352399 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:20.352412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.353225 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.353228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:20.354240 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:20.354249 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.354735 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.354739 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:20.354895 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:20.355513 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:25:20.355515 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:20.355554 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:25:20.355616 (+ 62us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:25:20.355914 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:25:20.355924 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:25:20.355931 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:25:20.356072 (+ 141us) client_negotiation.cc:240] Negotiation successful
0619 12:25:20.356347 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":143,"threads_started":1}
I20260619 12:25:20.358350 16503 heartbeater.cc:347] Connected to a master server at 127.6.231.254:44801
I20260619 12:25:20.358608 16503 heartbeater.cc:464] Registering TS with master...
I20260619 12:25:20.359129 16503 heartbeater.cc:511] Master 127.6.231.254:44801 requested a full tablet report, sending...
I20260619 12:25:20.360580 16315 ts_manager.cc:194] Registered new tserver with Master: 799f1215ca4442e1b539487bbd8b6209 (127.6.231.193:38419)
I20260619 12:25:20.361774 16315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.231.193:39131
W20260619 12:25:20.461823 16506 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:20.462123 16506 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:20.462201 16506 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:20.466223 16506 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:20.466297 16506 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:20.466382 16506 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:25:20.471243 16506 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:43077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:44801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16506
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:20.472332 16506 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:20.473801 16506 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:20.483273 16511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:20.483325 16512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:20.483389 16514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:20.483392 16506 server_base.cc:1034] running on GCE node
I20260619 12:25:20.483911 16506 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:20.484553 16506 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:20.485714 16506 hybrid_clock.cc:648] HybridClock initialized: now 1781871920485699 us; error 28 us; skew 500 ppm
I20260619 12:25:20.487567 16506 webserver.cc:460] Webserver started at http://127.6.231.194:43663/ using document root <none> and password file <none>
I20260619 12:25:20.488137 16506 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:20.488196 16506 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:20.488427 16506 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:20.490489 16506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "b6f07b93b2c042adbaf081f9f9e15d85"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "5dcba79940dec80c383ba55f374e306e"
server_key_iv: "2847f976375123213422f12d50579918"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.491118 16506 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "b6f07b93b2c042adbaf081f9f9e15d85"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "5dcba79940dec80c383ba55f374e306e"
server_key_iv: "2847f976375123213422f12d50579918"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.494745 16506 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:25:20.497239 16520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:20.498523 16506 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:25:20.498653 16506 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "b6f07b93b2c042adbaf081f9f9e15d85"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "5dcba79940dec80c383ba55f374e306e"
server_key_iv: "2847f976375123213422f12d50579918"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.498765 16506 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:20.530773 16506 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:20.531551 16506 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:20.531769 16506 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:20.532406 16506 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:20.533380 16506 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:20.533470 16506 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:20.533565 16506 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:20.533617 16506 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:20.542852 16506 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:43053
I20260619 12:25:20.542888 16633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:43053 every 8 connection(s)
I20260619 12:25:20.543835 16506 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:25:20.547391 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 16506
I20260619 12:25:20.547528 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:25:20.547854 7071 external_mini_cluster.cc:1428] Setting key 77e18db36af4e22612118f751d641a44
I20260619 12:25:20.550069 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:44801
--builtin_ntp_servers=127.6.231.212:43077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260619 12:25:20.555889 16354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:20.545783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:40967 (local address 127.6.231.254:44801)
0619 12:25:20.545929 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:20.545933 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:20.547168 (+ 1235us) server_negotiation.cc:407] Connection header received
0619 12:25:20.549398 (+ 2230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:20.549400 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:20.549502 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:20.549567 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:20.551043 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.551811 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.553222 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.553711 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.553970 (+ 259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:20.555159 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:20.555166 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:20.555172 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:25:20.555192 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:20.555220 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:25:20.555236 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:20.555339 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:20.555542 (+ 203us) server_negotiation.cc:299] Negotiation successful
0619 12:25:20.555632 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 12:25:20.556607 16636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:20.546214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:44801 (local address 127.6.231.194:40967)
0619 12:25:20.546966 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:20.547011 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:20.548988 (+ 1977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:20.549832 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:20.549845 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:20.549974 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:20.550791 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:20.550875 (+ 84us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.551980 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.551984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:20.553075 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:20.553085 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.553881 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.553884 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:20.554100 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:20.554862 (+ 762us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:25:20.554863 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:20.554907 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:25:20.554972 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:25:20.555344 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:25:20.555352 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:25:20.555356 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:25:20.555492 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 12:25:20.555726 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":118,"threads_started":1}
I20260619 12:25:20.557862 16634 heartbeater.cc:347] Connected to a master server at 127.6.231.254:44801
I20260619 12:25:20.558096 16634 heartbeater.cc:464] Registering TS with master...
I20260619 12:25:20.558621 16634 heartbeater.cc:511] Master 127.6.231.254:44801 requested a full tablet report, sending...
I20260619 12:25:20.559700 16315 ts_manager.cc:194] Registered new tserver with Master: b6f07b93b2c042adbaf081f9f9e15d85 (127.6.231.194:43053)
I20260619 12:25:20.560360 16315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.231.194:40967
W20260619 12:25:20.668007 16637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:20.668275 16637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:20.668314 16637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:20.672494 16637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:20.672593 16637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:20.672673 16637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:25:20.677496 16637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:43077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:44801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:20.678586 16637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:20.680215 16637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:20.689843 16643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:20.689845 16642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:20.690011 16645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:20.690157 16637 server_base.cc:1034] running on GCE node
I20260619 12:25:20.690696 16637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:20.691419 16637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:20.692632 16637 hybrid_clock.cc:648] HybridClock initialized: now 1781871920692602 us; error 52 us; skew 500 ppm
I20260619 12:25:20.694883 16637 webserver.cc:460] Webserver started at http://127.6.231.195:40685/ using document root <none> and password file <none>
I20260619 12:25:20.695472 16637 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:20.695561 16637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:20.695801 16637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:20.697952 16637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "dcad1e46a84d493ebbecadc91a357bfb"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "23c147fee1331420dc1eda25d83761c4"
server_key_iv: "5ec904745b0ed6d635eb69f3b5ce70b4"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.698524 16637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "dcad1e46a84d493ebbecadc91a357bfb"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "23c147fee1331420dc1eda25d83761c4"
server_key_iv: "5ec904745b0ed6d635eb69f3b5ce70b4"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.702558 16637 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 12:25:20.705271 16651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:20.706615 16637 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260619 12:25:20.706849 16637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "dcad1e46a84d493ebbecadc91a357bfb"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "23c147fee1331420dc1eda25d83761c4"
server_key_iv: "5ec904745b0ed6d635eb69f3b5ce70b4"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.706970 16637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:20.735378 16637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:20.736155 16637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:20.736369 16637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:20.736984 16637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:20.737960 16637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:20.738029 16637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:20.738101 16637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:20.738142 16637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:20.746697 16637 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:33149
I20260619 12:25:20.746726 16764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:33149 every 8 connection(s)
I20260619 12:25:20.747597 16637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:25:20.756932 16354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:20.749513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:46871 (local address 127.6.231.254:44801)
0619 12:25:20.749644 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:20.749648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:20.750726 (+ 1078us) server_negotiation.cc:407] Connection header received
0619 12:25:20.752167 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:20.752171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:20.752230 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:20.752332 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:20.753227 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.753830 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.754956 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.755197 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.755310 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:20.756435 (+ 1125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:20.756438 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:20.756440 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:25:20.756454 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:20.756472 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:25:20.756480 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:20.756554 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:20.756722 (+ 168us) server_negotiation.cc:299] Negotiation successful
0619 12:25:20.756785 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:25:20.757377 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 16637
I20260619 12:25:20.757509 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:25:20.757788 7071 external_mini_cluster.cc:1428] Setting key 09eb6dd4cb193e0af634f00ff21d4bee
I20260619 12:25:20.758049 16767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:20.749953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:44801 (local address 127.6.231.195:46871)
0619 12:25:20.750577 (+ 624us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:20.750616 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:20.751943 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:20.752506 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:20.752513 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:20.752597 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:20.753079 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:20.753090 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.753953 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.753958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:20.754820 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:20.754828 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:20.755346 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:20.755350 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:20.755514 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:20.756128 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:25:20.756129 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:20.756168 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:25:20.756233 (+ 65us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:25:20.756572 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:25:20.756580 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:25:20.756586 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:25:20.756710 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 12:25:20.756992 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":106,"threads_started":1}
I20260619 12:25:20.759343 16765 heartbeater.cc:347] Connected to a master server at 127.6.231.254:44801
I20260619 12:25:20.759583 16765 heartbeater.cc:464] Registering TS with master...
I20260619 12:25:20.760176 16765 heartbeater.cc:511] Master 127.6.231.254:44801 requested a full tablet report, sending...
I20260619 12:25:20.761222 16314 ts_manager.cc:194] Registered new tserver with Master: dcad1e46a84d493ebbecadc91a357bfb (127.6.231.195:33149)
I20260619 12:25:20.761945 16314 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.231.195:46871
I20260619 12:25:20.771970 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 12:25:20.774818 16354 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:20.773926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49926 (local address 127.6.231.254:44801)
0619 12:25:20.774079 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:20.774085 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:20.774199 (+ 114us) server_negotiation.cc:407] Connection header received
0619 12:25:20.774295 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:20.774297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:20.774337 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:20.774414 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:20.774671 (+ 257us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49926: BlockingRecv error: recv got EOF from 127.0.0.1:49926 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 12:25:20.774881 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 16375
I20260619 12:25:20.780036 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 16506
I20260619 12:25:20.785589 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 16637
I20260619 12:25:20.790659 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 16284
2026-06-19T12:25:20Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (903 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-06-19T12:25:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:25:20Z Disabled control of system clock
I20260619 12:25:20.819438 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:46551
--webserver_interface=127.6.231.254
--webserver_port=0
--builtin_ntp_servers=127.6.231.212:40557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:46551
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 12:25:20.928884 16776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:20.929157 16776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:20.929193 16776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:20.933059 16776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:25:20.933120 16776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:20.933136 16776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:20.933152 16776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:25:20.933167 16776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:25:20.937800 16776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:40557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:46551
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:46551
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:20.938834 16776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:20.940161 16776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:20.948546 16784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:20.948868 16781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:20.948551 16782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:20.948899 16776 server_base.cc:1034] running on GCE node
I20260619 12:25:20.949594 16776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:20.950640 16776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:20.951795 16776 hybrid_clock.cc:648] HybridClock initialized: now 1781871920951761 us; error 53 us; skew 500 ppm
I20260619 12:25:20.953722 16776 webserver.cc:460] Webserver started at http://127.6.231.254:34063/ using document root <none> and password file <none>
I20260619 12:25:20.954289 16776 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:20.954370 16776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:20.954584 16776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:20.956576 16776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "3705e2da68ec4f8f941e60d6ce8aa0d9"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "b7dba5608630a0abc42de77e4ce9db16"
server_key_iv: "c06a52b5036fbe73bd860a99e1ebe839"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.957127 16776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "3705e2da68ec4f8f941e60d6ce8aa0d9"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "b7dba5608630a0abc42de77e4ce9db16"
server_key_iv: "c06a52b5036fbe73bd860a99e1ebe839"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.960775 16776 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260619 12:25:20.963188 16790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:20.964393 16776 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:25:20.964540 16776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "3705e2da68ec4f8f941e60d6ce8aa0d9"
format_stamp: "Formatted at 2026-06-19 12:25:20 on dist-test-slave-kr9h"
server_key: "b7dba5608630a0abc42de77e4ce9db16"
server_key_iv: "c06a52b5036fbe73bd860a99e1ebe839"
server_key_version: "encryptionkey@0"
I20260619 12:25:20.964644 16776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:20.969151 16776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:20.969791 16776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:20.969965 16776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:20.976544 16776 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:46551
I20260619 12:25:20.976594 16842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:46551 every 8 connection(s)
I20260619 12:25:20.977651 16776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:25:20.981202 16843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:25:20.985021 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 16776
I20260619 12:25:20.985162 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:25:20.985482 7071 external_mini_cluster.cc:1428] Setting key 9df18f4aac1a8a81ee07cd5466c3f13c
I20260619 12:25:20.987579 16843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9: Bootstrap starting.
I20260619 12:25:20.989984 16843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9: Neither blocks nor log segments found. Creating new log.
I20260619 12:25:20.990828 16843 log.cc:824] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9: Log is configured to *not* fsync() on all Append() calls
I20260619 12:25:20.991554 16846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:20.986999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44238 (local address 127.6.231.254:46551)
0619 12:25:20.987727 (+ 728us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:20.987745 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:20.987783 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:25:20.988455 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:20.988472 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:20.988839 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:20.989197 (+ 358us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:20.990016 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:20.990021 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:20.990052 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:25:20.990065 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:20.990133 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:25:20.990159 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:20.990267 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:20.990405 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 12:25:20.990654 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":139,"threads_started":1}
I20260619 12:25:20.992784 16843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9: No bootstrap required, opened a new log
I20260619 12:25:20.996353 16843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3705e2da68ec4f8f941e60d6ce8aa0d9" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46551 } }
I20260619 12:25:20.996587 16843 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:25:20.996657 16843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3705e2da68ec4f8f941e60d6ce8aa0d9, State: Initialized, Role: FOLLOWER
I20260619 12:25:20.997071 16843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3705e2da68ec4f8f941e60d6ce8aa0d9" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46551 } }
I20260619 12:25:20.997213 16843 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:25:20.997275 16843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:25:20.997406 16843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:20.998538 16843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3705e2da68ec4f8f941e60d6ce8aa0d9" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46551 } }
I20260619 12:25:20.998891 16843 leader_election.cc:302] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3705e2da68ec4f8f941e60d6ce8aa0d9; no voters:
I20260619 12:25:20.999202 16843 leader_election.cc:288] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:25:20.999336 16848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:25:20.999576 16848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [term 1 LEADER]: Becoming Leader. State: Replica: 3705e2da68ec4f8f941e60d6ce8aa0d9, State: Running, Role: LEADER
I20260619 12:25:20.999905 16848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3705e2da68ec4f8f941e60d6ce8aa0d9" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46551 } }
I20260619 12:25:21.000495 16843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:25:21.001605 16850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3705e2da68ec4f8f941e60d6ce8aa0d9. Latest consensus state: current_term: 1 leader_uuid: "3705e2da68ec4f8f941e60d6ce8aa0d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3705e2da68ec4f8f941e60d6ce8aa0d9" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46551 } } }
I20260619 12:25:21.001708 16850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [sys.catalog]: This master's current role is: LEADER
I20260619 12:25:21.001362 16849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3705e2da68ec4f8f941e60d6ce8aa0d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3705e2da68ec4f8f941e60d6ce8aa0d9" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46551 } } }
I20260619 12:25:21.002002 16849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9 [sys.catalog]: This master's current role is: LEADER
I20260619 12:25:21.002202 16857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:25:21.005189 16857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:25:21.010528 16857 catalog_manager.cc:1347] Generated new cluster ID: 53739c1a5e7d40848a6e2c7b79606e3f
I20260619 12:25:21.010619 16857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:25:21.023442 16857 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:25:21.023766 16857 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:25:21.031636 16857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3705e2da68ec4f8f941e60d6ce8aa0d9: Generated new TSK 0
I20260619 12:25:21.032380 16857 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:25:21.039908 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:46551
--builtin_ntp_servers=127.6.231.212:40557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 12:25:21.159463 16867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:21.159710 16867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:21.159752 16867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:21.163848 16867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:21.163904 16867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:21.163995 16867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:25:21.168907 16867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:40557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:46551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:21.169986 16867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:21.171352 16867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:21.181237 16872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:21.181236 16873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:21.181262 16875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:21.181656 16867 server_base.cc:1034] running on GCE node
I20260619 12:25:21.182092 16867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:21.182729 16867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:21.184046 16867 hybrid_clock.cc:648] HybridClock initialized: now 1781871921184036 us; error 30 us; skew 500 ppm
I20260619 12:25:21.186116 16867 webserver.cc:460] Webserver started at http://127.6.231.193:32797/ using document root <none> and password file <none>
I20260619 12:25:21.186681 16867 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:21.186769 16867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:21.187011 16867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:21.189033 16867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "ea0e2fd01ec04de9ae5b44ea87463655"
format_stamp: "Formatted at 2026-06-19 12:25:21 on dist-test-slave-kr9h"
server_key: "cc0b50222748f9702d4216049a45d81e"
server_key_iv: "4f8e76f7c759aab135ff59aa4e9d13f5"
server_key_version: "encryptionkey@0"
I20260619 12:25:21.189630 16867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "ea0e2fd01ec04de9ae5b44ea87463655"
format_stamp: "Formatted at 2026-06-19 12:25:21 on dist-test-slave-kr9h"
server_key: "cc0b50222748f9702d4216049a45d81e"
server_key_iv: "4f8e76f7c759aab135ff59aa4e9d13f5"
server_key_version: "encryptionkey@0"
I20260619 12:25:21.193075 16867 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:25:21.195621 16881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:21.197052 16867 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:25:21.197181 16867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "ea0e2fd01ec04de9ae5b44ea87463655"
format_stamp: "Formatted at 2026-06-19 12:25:21 on dist-test-slave-kr9h"
server_key: "cc0b50222748f9702d4216049a45d81e"
server_key_iv: "4f8e76f7c759aab135ff59aa4e9d13f5"
server_key_version: "encryptionkey@0"
I20260619 12:25:21.197343 16867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:21.202914 16867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:21.203511 16867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:21.203694 16867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:21.204217 16867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:21.205092 16867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:21.205163 16867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:21.205230 16867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:21.205289 16867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:21.214268 16867 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:34803
I20260619 12:25:21.214283 16994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:34803 every 8 connection(s)
I20260619 12:25:21.215201 16867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:25:21.216599 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 16867
I20260619 12:25:21.216712 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:25:21.216998 7071 external_mini_cluster.cc:1428] Setting key e6217a080d62d35a07683c2eb06ff234
I20260619 12:25:21.219197 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:46551
--builtin_ntp_servers=127.6.231.212:40557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260619 12:25:21.222622 16846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:21.217228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:48621 (local address 127.6.231.254:46551)
0619 12:25:21.217382 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:21.217387 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:21.218723 (+ 1336us) server_negotiation.cc:407] Connection header received
0619 12:25:21.220663 (+ 1940us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:21.220668 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:21.220718 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:21.220827 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:21.222100 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:21.222107 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:21.222113 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:25:21.222129 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:21.222155 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:25:21.222168 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:21.222272 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:21.222389 (+ 117us) server_negotiation.cc:299] Negotiation successful
0619 12:25:21.222436 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:25:21.223429 16997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:21.217763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:46551 (local address 127.6.231.193:48621)
0619 12:25:21.218560 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:21.218617 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:21.220402 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:21.220990 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:21.220997 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:21.221122 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:21.221815 (+ 693us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:25:21.221816 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:21.221876 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:25:21.221949 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:25:21.222250 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:25:21.222254 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:25:21.222258 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:25:21.222378 (+ 120us) client_negotiation.cc:240] Negotiation successful
0619 12:25:21.222565 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":138,"threads_started":1}
I20260619 12:25:21.224640 16995 heartbeater.cc:347] Connected to a master server at 127.6.231.254:46551
I20260619 12:25:21.224855 16995 heartbeater.cc:464] Registering TS with master...
I20260619 12:25:21.225288 16995 heartbeater.cc:511] Master 127.6.231.254:46551 requested a full tablet report, sending...
I20260619 12:25:21.226778 16807 ts_manager.cc:194] Registered new tserver with Master: ea0e2fd01ec04de9ae5b44ea87463655 (127.6.231.193:34803)
W20260619 12:25:21.333738 16998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:21.334023 16998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:21.334065 16998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:21.338178 16998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:21.338238 16998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:21.338294 16998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:25:21.343016 16998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:40557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:46551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:21.344105 16998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:21.345664 16998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:21.355240 17003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:21.355427 16998 server_base.cc:1034] running on GCE node
W20260619 12:25:21.355250 17004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:21.355309 17006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:21.355908 16998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:21.356523 16998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:21.357677 16998 hybrid_clock.cc:648] HybridClock initialized: now 1781871921357651 us; error 40 us; skew 500 ppm
I20260619 12:25:21.359541 16998 webserver.cc:460] Webserver started at http://127.6.231.194:35839/ using document root <none> and password file <none>
I20260619 12:25:21.360108 16998 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:21.360172 16998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:21.360430 16998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:21.362505 16998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "3ee201c740114f8081350867b38645e0"
format_stamp: "Formatted at 2026-06-19 12:25:21 on dist-test-slave-kr9h"
server_key: "03da627bd3d7c013992065d43d9a06ca"
server_key_iv: "03c339f45e43d6b2757a42d2554314df"
server_key_version: "encryptionkey@0"
I20260619 12:25:21.363051 16998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "3ee201c740114f8081350867b38645e0"
format_stamp: "Formatted at 2026-06-19 12:25:21 on dist-test-slave-kr9h"
server_key: "03da627bd3d7c013992065d43d9a06ca"
server_key_iv: "03c339f45e43d6b2757a42d2554314df"
server_key_version: "encryptionkey@0"
I20260619 12:25:21.366734 16998 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:25:21.369519 17012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:21.370782 16998 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:25:21.370931 16998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "3ee201c740114f8081350867b38645e0"
format_stamp: "Formatted at 2026-06-19 12:25:21 on dist-test-slave-kr9h"
server_key: "03da627bd3d7c013992065d43d9a06ca"
server_key_iv: "03c339f45e43d6b2757a42d2554314df"
server_key_version: "encryptionkey@0"
I20260619 12:25:21.371063 16998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:21.375663 16998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:21.376324 16998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:21.376513 16998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:21.377063 16998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:21.378046 16998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:21.378123 16998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:21.378191 16998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:21.378222 16998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:21.387143 16998 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:45137
I20260619 12:25:21.387187 17125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:45137 every 8 connection(s)
I20260619 12:25:21.388077 16998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:25:21.394321 16846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:21.389968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:46733 (local address 127.6.231.254:46551)
0619 12:25:21.390125 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:21.390128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:21.391208 (+ 1080us) server_negotiation.cc:407] Connection header received
0619 12:25:21.392567 (+ 1359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:21.392571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:21.392616 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:21.392697 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:21.393831 (+ 1134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:21.393835 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:21.393837 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:25:21.393850 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:21.393871 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:25:21.393880 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:21.393986 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:21.394130 (+ 144us) server_negotiation.cc:299] Negotiation successful
0619 12:25:21.394175 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 12:25:21.395112 17128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:21.390373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:46551 (local address 127.6.231.194:46733)
0619 12:25:21.391045 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:21.391087 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:21.392352 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:21.392860 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:21.392867 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:21.392955 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:21.393570 (+ 615us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:25:21.393572 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:21.393610 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:25:21.393677 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:25:21.393971 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:25:21.393979 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:25:21.393984 (+ 5us) client_negotiation.cc:769] Sending connection context
0619 12:25:21.394161 (+ 177us) client_negotiation.cc:240] Negotiation successful
0619 12:25:21.394403 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":115,"threads_started":1}
I20260619 12:25:21.396044 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 16998
I20260619 12:25:21.396174 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:25:21.396394 17126 heartbeater.cc:347] Connected to a master server at 127.6.231.254:46551
I20260619 12:25:21.396548 7071 external_mini_cluster.cc:1428] Setting key 29f04851f9fdea39b30a4ffe17b02ce0
I20260619 12:25:21.396627 17126 heartbeater.cc:464] Registering TS with master...
I20260619 12:25:21.397063 17126 heartbeater.cc:511] Master 127.6.231.254:46551 requested a full tablet report, sending...
I20260619 12:25:21.398147 16807 ts_manager.cc:194] Registered new tserver with Master: 3ee201c740114f8081350867b38645e0 (127.6.231.194:45137)
I20260619 12:25:21.398767 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:46551
--builtin_ntp_servers=127.6.231.212:40557
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260619 12:25:21.509792 17129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:21.510052 17129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:21.510092 17129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:21.514117 17129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:21.514181 17129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:21.514240 17129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:25:21.519070 17129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:40557
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:46551
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:21.520131 17129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:21.521564 17129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:21.531092 17134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:21.531096 17137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:21.531131 17129 server_base.cc:1034] running on GCE node
W20260619 12:25:21.531092 17135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:21.531836 17129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:21.532512 17129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:21.533679 17129 hybrid_clock.cc:648] HybridClock initialized: now 1781871921533657 us; error 37 us; skew 500 ppm
I20260619 12:25:21.535773 17129 webserver.cc:460] Webserver started at http://127.6.231.195:37345/ using document root <none> and password file <none>
I20260619 12:25:21.536406 17129 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:21.536476 17129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:21.536707 17129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:21.539122 17129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "5f764fb8596f4a0aa112be2c5383b772"
format_stamp: "Formatted at 2026-06-19 12:25:21 on dist-test-slave-kr9h"
server_key: "81efccdccd5f5935e250103bebc25c02"
server_key_iv: "80d3eee324a4a877e6d837c8b1f8a243"
server_key_version: "encryptionkey@0"
I20260619 12:25:21.539726 17129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "5f764fb8596f4a0aa112be2c5383b772"
format_stamp: "Formatted at 2026-06-19 12:25:21 on dist-test-slave-kr9h"
server_key: "81efccdccd5f5935e250103bebc25c02"
server_key_iv: "80d3eee324a4a877e6d837c8b1f8a243"
server_key_version: "encryptionkey@0"
I20260619 12:25:21.543697 17129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:25:21.546627 17143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:21.548023 17129 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:25:21.548175 17129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "5f764fb8596f4a0aa112be2c5383b772"
format_stamp: "Formatted at 2026-06-19 12:25:21 on dist-test-slave-kr9h"
server_key: "81efccdccd5f5935e250103bebc25c02"
server_key_iv: "80d3eee324a4a877e6d837c8b1f8a243"
server_key_version: "encryptionkey@0"
I20260619 12:25:21.548300 17129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:21.552922 17129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:21.553589 17129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:21.553767 17129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:21.554291 17129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:21.555171 17129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:21.555240 17129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:21.555310 17129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:21.555363 17129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:21.563913 17129 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:41975
I20260619 12:25:21.563939 17256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:41975 every 8 connection(s)
I20260619 12:25:21.564814 17129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:25:21.565310 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 17129
I20260619 12:25:21.565420 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:25:21.565765 7071 external_mini_cluster.cc:1428] Setting key abc5e6f6e775731fc87a3a11c1e87628
I20260619 12:25:21.571758 16846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:21.566690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:42013 (local address 127.6.231.254:46551)
0619 12:25:21.566858 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:21.566862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:21.568419 (+ 1557us) server_negotiation.cc:407] Connection header received
0619 12:25:21.570022 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:21.570025 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:21.570071 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:21.570157 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:21.571240 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:21.571244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:21.571247 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:25:21.571260 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:21.571283 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:25:21.571293 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:21.571413 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:21.571567 (+ 154us) server_negotiation.cc:299] Negotiation successful
0619 12:25:21.571615 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260619 12:25:21.572646 17259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:21.567295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:46551 (local address 127.6.231.195:42013)
0619 12:25:21.568203 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:21.568263 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:21.569793 (+ 1530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:21.570336 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:21.570344 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:21.570438 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:21.570972 (+ 534us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:25:21.570974 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:21.571016 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:25:21.571087 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:25:21.571400 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:25:21.571409 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:25:21.571415 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:25:21.571613 (+ 198us) client_negotiation.cc:240] Negotiation successful
0619 12:25:21.571866 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":124,"threads_started":1}
I20260619 12:25:21.573909 17257 heartbeater.cc:347] Connected to a master server at 127.6.231.254:46551
I20260619 12:25:21.574168 17257 heartbeater.cc:464] Registering TS with master...
I20260619 12:25:21.574600 17257 heartbeater.cc:511] Master 127.6.231.254:46551 requested a full tablet report, sending...
I20260619 12:25:21.575678 16807 ts_manager.cc:194] Registered new tserver with Master: 5f764fb8596f4a0aa112be2c5383b772 (127.6.231.195:41975)
I20260619 12:25:21.580179 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260619 12:25:21.583105 16846 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:25:21.582184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44252 (local address 127.6.231.254:46551)
0619 12:25:21.582323 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:21.582327 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:21.582603 (+ 276us) server_negotiation.cc:407] Connection header received
0619 12:25:21.582643 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:21.582645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:21.582677 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:21.582738 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:21.582945 (+ 207us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44252: BlockingRecv error: recv got EOF from 127.0.0.1:44252 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 12:25:21.583585 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 16867
I20260619 12:25:21.589200 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 16998
I20260619 12:25:21.594578 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 17129
I20260619 12:25:21.599802 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 16776
2026-06-19T12:25:21Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (809 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:25:21 dist-test-slave-kr9h krb5kdc[17267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:25:21 dist-test-slave-kr9h krb5kdc[17267](info): set up 2 sockets
Jun 19 12:25:21 dist-test-slave-kr9h krb5kdc[17267](info): commencing operation
krb5kdc: starting...
W20260619 12:25:22.642330 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:25:22 dist-test-slave-kr9h krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:25:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:25:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:25:22.770687 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:36749
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:32953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:36749
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:25:22.897234 17280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:22.897698 17280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:22.897778 17280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:22.905264 17280 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:25:22.905370 17280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:22.905409 17280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:22.905469 17280 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:25:22.905498 17280 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:25:22.911329 17280 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:32953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:36749
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:36749
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:22.912539 17280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:22.913693 17280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:22.922473 17285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:22.922479 17286 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:22.922585 17280 server_base.cc:1034] running on GCE node
W20260619 12:25:22.922477 17288 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:22.923293 17280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:22.924409 17280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:22.925558 17280 hybrid_clock.cc:648] HybridClock initialized: now 1781871922925524 us; error 75 us; skew 500 ppm
Jun 19 12:25:22 dist-test-slave-kr9h krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871922, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:25:22.928401 17280 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:25:22.929410 17280 webserver.cc:460] Webserver started at http://127.6.231.254:41119/ using document root <none> and password file <none>
I20260619 12:25:22.930063 17280 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:22.930126 17280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:22.930371 17280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:22.932463 17280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "521787f9c3a94b66b2042360f0e42aba"
format_stamp: "Formatted at 2026-06-19 12:25:22 on dist-test-slave-kr9h"
server_key: "0ca5ea1ee7f2aa85e72e27fae2e57098"
server_key_iv: "a456bbcfc4adad77d59d3e5d9d5ab805"
server_key_version: "encryptionkey@0"
I20260619 12:25:22.933032 17280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "521787f9c3a94b66b2042360f0e42aba"
format_stamp: "Formatted at 2026-06-19 12:25:22 on dist-test-slave-kr9h"
server_key: "0ca5ea1ee7f2aa85e72e27fae2e57098"
server_key_iv: "a456bbcfc4adad77d59d3e5d9d5ab805"
server_key_version: "encryptionkey@0"
I20260619 12:25:22.936933 17280 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 12:25:22.939523 17295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:22.940863 17280 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:25:22.941012 17280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "521787f9c3a94b66b2042360f0e42aba"
format_stamp: "Formatted at 2026-06-19 12:25:22 on dist-test-slave-kr9h"
server_key: "0ca5ea1ee7f2aa85e72e27fae2e57098"
server_key_iv: "a456bbcfc4adad77d59d3e5d9d5ab805"
server_key_version: "encryptionkey@0"
I20260619 12:25:22.941131 17280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:22.967468 17280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:22.970768 17280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:22.970976 17280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:22.978545 17280 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:36749
I20260619 12:25:22.978845 17347 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:36749 every 8 connection(s)
I20260619 12:25:22.979642 17280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:25:22.982515 17348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:25:22.988140 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 17280
I20260619 12:25:22.988118 17348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba: Bootstrap starting.
I20260619 12:25:22.988261 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:25:22.988560 7071 external_mini_cluster.cc:1428] Setting key 268fc034cdd880afcd040dd0c8cf5ab2
I20260619 12:25:22.991124 17348 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba: Neither blocks nor log segments found. Creating new log.
I20260619 12:25:22.991869 17348 log.cc:824] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba: Log is configured to *not* fsync() on all Append() calls
I20260619 12:25:22.993883 17348 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba: No bootstrap required, opened a new log
I20260619 12:25:22.997330 17348 raft_consensus.cc:348] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521787f9c3a94b66b2042360f0e42aba" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36749 } }
I20260619 12:25:22.997650 17348 raft_consensus.cc:374] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:25:22.997741 17348 raft_consensus.cc:729] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 521787f9c3a94b66b2042360f0e42aba, State: Initialized, Role: FOLLOWER
I20260619 12:25:22.998202 17348 consensus_queue.cc:260] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521787f9c3a94b66b2042360f0e42aba" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36749 } }
I20260619 12:25:22.998358 17348 raft_consensus.cc:388] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:25:22.998443 17348 raft_consensus.cc:482] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:25:22.998544 17348 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:22.999512 17348 raft_consensus.cc:504] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521787f9c3a94b66b2042360f0e42aba" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36749 } }
I20260619 12:25:22.999861 17348 leader_election.cc:302] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 521787f9c3a94b66b2042360f0e42aba; no voters:
I20260619 12:25:23.000177 17348 leader_election.cc:288] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:25:23.000617 17354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521787f9c3a94b66b2042360f0e42aba" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36749 } } }
I20260619 12:25:23.000751 17354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:25:23.000941 17353 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:25:23.001077 17353 raft_consensus.cc:686] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [term 1 LEADER]: Becoming Leader. State: Replica: 521787f9c3a94b66b2042360f0e42aba, State: Running, Role: LEADER
I20260619 12:25:23.001392 17353 consensus_queue.cc:237] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521787f9c3a94b66b2042360f0e42aba" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36749 } }
I20260619 12:25:23.001569 17348 sys_catalog.cc:564] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [sys.catalog]: configured and running, proceeding with master startup.
Jun 19 12:25:22 dist-test-slave-kr9h krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:25:23.003031 17354 sys_catalog.cc:455] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 521787f9c3a94b66b2042360f0e42aba. Latest consensus state: current_term: 1 leader_uuid: "521787f9c3a94b66b2042360f0e42aba" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "521787f9c3a94b66b2042360f0e42aba" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36749 } } }
I20260619 12:25:23.003150 17354 sys_catalog.cc:458] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba [sys.catalog]: This master's current role is: LEADER
I20260619 12:25:23.003819 17361 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:25:23.007184 17361 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:25:23.008831 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:22.990161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45980 (local address 127.6.231.254:36749)
0619 12:25:22.990726 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:22.990741 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:22.990776 (+ 35us) server_negotiation.cc:407] Connection header received
0619 12:25:22.991476 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:22.991494 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:22.991887 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:22.992272 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:22.993091 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:22.994196 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:22.995028 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:22.995493 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:22.995663 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.002941 (+ 7278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:23.002949 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:23.002967 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:25:23.003002 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:23.005507 (+ 2505us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:23.006053 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:23.006058 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:23.006064 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:23.006204 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:23.006506 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:23.006509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:23.006511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:23.007014 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:23.007224 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:23.007510 (+ 286us) server_negotiation.cc:299] Negotiation successful
0619 12:25:23.007827 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":218,"threads_started":1}
I20260619 12:25:23.015481 17361 catalog_manager.cc:1347] Generated new cluster ID: 2d8d543f1453494497e65916a440606c
I20260619 12:25:23.015580 17361 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:25:23.024631 17361 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:25:23.025691 17361 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:25:23.034350 17361 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba: Generated new TSK 0
I20260619 12:25:23.035207 17361 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:25:23.070029 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36749
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:32953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:25:23.181308 17373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:23.181665 17373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:23.181748 17373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:23.185813 17373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:23.185894 17373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:23.185987 17373 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:25:23.191329 17373 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:32953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:23.192363 17373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:23.193396 17373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:23.203251 17379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:23.203233 17381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:23.203289 17378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:23.203459 17373 server_base.cc:1034] running on GCE node
I20260619 12:25:23.203960 17373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:23.204778 17373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:23.206347 17373 hybrid_clock.cc:648] HybridClock initialized: now 1781871923206261 us; error 124 us; skew 500 ppm
Jun 19 12:25:23 dist-test-slave-kr9h krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871923, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:25:23.209229 17373 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:25:23.210144 17373 webserver.cc:460] Webserver started at http://127.6.231.193:42007/ using document root <none> and password file <none>
I20260619 12:25:23.210713 17373 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:23.210803 17373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:23.211006 17373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:23.213037 17373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "622cbdb76f3f49a58f07a5491e4dcfa9"
format_stamp: "Formatted at 2026-06-19 12:25:23 on dist-test-slave-kr9h"
server_key: "64fa311dc0663f743757228b9e8b82b9"
server_key_iv: "577b117f94aee526fef4a2355c431595"
server_key_version: "encryptionkey@0"
I20260619 12:25:23.213658 17373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "622cbdb76f3f49a58f07a5491e4dcfa9"
format_stamp: "Formatted at 2026-06-19 12:25:23 on dist-test-slave-kr9h"
server_key: "64fa311dc0663f743757228b9e8b82b9"
server_key_iv: "577b117f94aee526fef4a2355c431595"
server_key_version: "encryptionkey@0"
I20260619 12:25:23.217353 17373 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:25:23.219945 17388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:23.221189 17373 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 12:25:23.221340 17373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "622cbdb76f3f49a58f07a5491e4dcfa9"
format_stamp: "Formatted at 2026-06-19 12:25:23 on dist-test-slave-kr9h"
server_key: "64fa311dc0663f743757228b9e8b82b9"
server_key_iv: "577b117f94aee526fef4a2355c431595"
server_key_version: "encryptionkey@0"
I20260619 12:25:23.221484 17373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:23.237601 17373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:23.240944 17373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:23.241150 17373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:23.241813 17373 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:23.242631 17373 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:23.242699 17373 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:23.242763 17373 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:23.242812 17373 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:23.252036 17373 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:34165
I20260619 12:25:23.252056 17501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:34165 every 8 connection(s)
I20260619 12:25:23.252920 17373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:25:23.256578 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 17373
I20260619 12:25:23.256721 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:25:23.257061 7071 external_mini_cluster.cc:1428] Setting key 4ed01b37ea4c155e1d7d08a1b4a1a893
Jun 19 12:25:23 dist-test-slave-kr9h krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871923, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
I20260619 12:25:23.269312 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.254952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:58711 (local address 127.6.231.254:36749)
0619 12:25:23.255088 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:23.255091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:23.255979 (+ 888us) server_negotiation.cc:407] Connection header received
0619 12:25:23.257777 (+ 1798us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:23.257781 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:23.257837 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:23.257923 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:23.259416 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.260054 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.261298 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.261594 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.261752 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.264947 (+ 3195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:23.264953 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:23.264957 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:25:23.264987 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:23.266783 (+ 1796us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:23.267399 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:23.267405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:23.267410 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:23.267511 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:23.267887 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:23.267894 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:23.267897 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:23.268167 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:23.268273 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:23.268907 (+ 634us) server_negotiation.cc:299] Negotiation successful
0619 12:25:23.269063 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:25:23.270146 17504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.255278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36749 (local address 127.6.231.193:58711)
0619 12:25:23.255813 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:23.255857 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:23.257559 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:23.258110 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:23.258117 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:23.258510 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:23.259226 (+ 716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.259250 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.260210 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.260212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.261171 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.261192 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.261817 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.261824 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.262053 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.262849 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:23.262873 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:23.264768 (+ 1895us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:25:23.266956 (+ 2188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:25:23.266964 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:25:23.266990 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:25:23.267299 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:25:23.267611 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:25:23.267613 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:25:23.267614 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:25:23.267784 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:25:23.268269 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:25:23.268274 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:25:23.268618 (+ 344us) client_negotiation.cc:769] Sending connection context
0619 12:25:23.268822 (+ 204us) client_negotiation.cc:240] Negotiation successful
0619 12:25:23.269103 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:25:23.271428 17502 heartbeater.cc:347] Connected to a master server at 127.6.231.254:36749
I20260619 12:25:23.271660 17502 heartbeater.cc:464] Registering TS with master...
I20260619 12:25:23.272228 17502 heartbeater.cc:511] Master 127.6.231.254:36749 requested a full tablet report, sending...
I20260619 12:25:23.274056 17312 ts_manager.cc:194] Registered new tserver with Master: 622cbdb76f3f49a58f07a5491e4dcfa9 (127.6.231.193:34165)
I20260619 12:25:23.275715 17312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:58711
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:25:23.288064 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36749
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:32953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:25:23.399165 17507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:23.399423 17507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:23.399467 17507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:23.403825 17507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:23.403884 17507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:23.403970 17507 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:25:23.409200 17507 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:32953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17507
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:23.410399 17507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:23.411478 17507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:23.421118 17513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:23.421197 17512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:23.421094 17515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:23.421270 17507 server_base.cc:1034] running on GCE node
I20260619 12:25:23.421761 17507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:23.422436 17507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:23.423597 17507 hybrid_clock.cc:648] HybridClock initialized: now 1781871923423561 us; error 50 us; skew 500 ppm
Jun 19 12:25:23 dist-test-slave-kr9h krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871923, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:25:23.426512 17507 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:25:23.427578 17507 webserver.cc:460] Webserver started at http://127.6.231.194:37941/ using document root <none> and password file <none>
I20260619 12:25:23.428184 17507 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:23.428251 17507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:23.428464 17507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:23.430706 17507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "ac2a0553fee949769d9c96acfcf3e24b"
format_stamp: "Formatted at 2026-06-19 12:25:23 on dist-test-slave-kr9h"
server_key: "bf55a3da8edc217863a1514793544ebe"
server_key_iv: "e22eb79c9ad6c4c2c2ba5a9188a1f164"
server_key_version: "encryptionkey@0"
I20260619 12:25:23.431293 17507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "ac2a0553fee949769d9c96acfcf3e24b"
format_stamp: "Formatted at 2026-06-19 12:25:23 on dist-test-slave-kr9h"
server_key: "bf55a3da8edc217863a1514793544ebe"
server_key_iv: "e22eb79c9ad6c4c2c2ba5a9188a1f164"
server_key_version: "encryptionkey@0"
I20260619 12:25:23.435055 17507 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 12:25:23.437549 17522 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:23.438783 17507 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:25:23.438907 17507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "ac2a0553fee949769d9c96acfcf3e24b"
format_stamp: "Formatted at 2026-06-19 12:25:23 on dist-test-slave-kr9h"
server_key: "bf55a3da8edc217863a1514793544ebe"
server_key_iv: "e22eb79c9ad6c4c2c2ba5a9188a1f164"
server_key_version: "encryptionkey@0"
I20260619 12:25:23.439047 17507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:23.451668 17507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:23.455051 17507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:23.455284 17507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:23.455885 17507 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:23.456744 17507 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:23.456817 17507 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:23.456888 17507 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:23.456933 17507 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:23.466332 17507 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:40485
I20260619 12:25:23.466352 17635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:40485 every 8 connection(s)
I20260619 12:25:23.467253 17507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:25:23.474342 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 17507
I20260619 12:25:23.474476 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:25:23.474779 7071 external_mini_cluster.cc:1428] Setting key 957f89f0a4f60b52498b7b6db97e6494
Jun 19 12:25:23 dist-test-slave-kr9h krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871923, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:25:23.481496 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.469092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:38077 (local address 127.6.231.254:36749)
0619 12:25:23.469265 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:23.469269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:23.470158 (+ 889us) server_negotiation.cc:407] Connection header received
0619 12:25:23.471584 (+ 1426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:23.471587 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:23.471641 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:23.471725 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:23.472919 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.473541 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.474745 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.475020 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.475131 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.478044 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:23.478048 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:23.478051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:25:23.478081 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:23.479489 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:23.479992 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:23.479997 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:23.479999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:23.480078 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:23.480413 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:23.480418 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:23.480420 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:23.480599 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:23.480701 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:23.481189 (+ 488us) server_negotiation.cc:299] Negotiation successful
0619 12:25:23.481303 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260619 12:25:23.482333 17638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.469549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36749 (local address 127.6.231.194:38077)
0619 12:25:23.469999 (+ 450us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:23.470041 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:23.471367 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:23.471939 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:23.471946 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:23.472293 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:23.472767 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.472779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.473662 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.473666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.474622 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.474631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.475169 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.475174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.475372 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.476098 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:23.476118 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:23.477885 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:25:23.479616 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:25:23.479622 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:25:23.479637 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:25:23.479889 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:25:23.480175 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:25:23.480179 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:25:23.480182 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:25:23.480321 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:25:23.480710 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:25:23.480718 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:25:23.480960 (+ 242us) client_negotiation.cc:769] Sending connection context
0619 12:25:23.481152 (+ 192us) client_negotiation.cc:240] Negotiation successful
0619 12:25:23.481506 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":113,"threads_started":1}
I20260619 12:25:23.483538 17636 heartbeater.cc:347] Connected to a master server at 127.6.231.254:36749
I20260619 12:25:23.483824 17636 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
I20260619 12:25:23.484364 17636 heartbeater.cc:511] Master 127.6.231.254:36749 requested a full tablet report, sending...
I20260619 12:25:23.485404 17312 ts_manager.cc:194] Registered new tserver with Master: ac2a0553fee949769d9c96acfcf3e24b (127.6.231.194:40485)
I20260619 12:25:23.485998 17312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:38077
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:25:23.504001 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36749
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:32953
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:25:23.621300 17641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:25:23.621645 17641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:25:23.621696 17641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:25:23.625836 17641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:25:23.625931 17641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:25:23.625993 17641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:25:23.631450 17641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:32953
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36749
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:25:23.632529 17641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:25:23.633571 17641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:25:23.643213 17647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:25:23.643245 17649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:23.643543 17641 server_base.cc:1034] running on GCE node
W20260619 12:25:23.643209 17646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:25:23.643967 17641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:25:23.644555 17641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:25:23.645799 17641 hybrid_clock.cc:648] HybridClock initialized: now 1781871923645756 us; error 56 us; skew 500 ppm
Jun 19 12:25:23 dist-test-slave-kr9h krb5kdc[17267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871923, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:25:23.648485 17641 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:25:23.649346 17641 webserver.cc:460] Webserver started at http://127.6.231.195:34229/ using document root <none> and password file <none>
I20260619 12:25:23.649963 17641 fs_manager.cc:359] Metadata directory not provided
I20260619 12:25:23.650024 17641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:25:23.650233 17641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:25:23.652252 17641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "b378c381832f43ec8f1ce482c780329b"
format_stamp: "Formatted at 2026-06-19 12:25:23 on dist-test-slave-kr9h"
server_key: "d64209682b12ae52e5aec8f7dfaa4f8f"
server_key_iv: "1a6fe79b057351770eb2ab3b56adb1ca"
server_key_version: "encryptionkey@0"
I20260619 12:25:23.652808 17641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "b378c381832f43ec8f1ce482c780329b"
format_stamp: "Formatted at 2026-06-19 12:25:23 on dist-test-slave-kr9h"
server_key: "d64209682b12ae52e5aec8f7dfaa4f8f"
server_key_iv: "1a6fe79b057351770eb2ab3b56adb1ca"
server_key_version: "encryptionkey@0"
I20260619 12:25:23.656549 17641 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.006s sys 0.000s
I20260619 12:25:23.659085 17656 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:23.660323 17641 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:25:23.660450 17641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "b378c381832f43ec8f1ce482c780329b"
format_stamp: "Formatted at 2026-06-19 12:25:23 on dist-test-slave-kr9h"
server_key: "d64209682b12ae52e5aec8f7dfaa4f8f"
server_key_iv: "1a6fe79b057351770eb2ab3b56adb1ca"
server_key_version: "encryptionkey@0"
I20260619 12:25:23.660558 17641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:25:23.677380 17641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:25:23.680823 17641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:25:23.681036 17641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:25:23.681700 17641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:25:23.682524 17641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:25:23.682605 17641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:23.682689 17641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:25:23.682740 17641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:25:23.691825 17641 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:41425
I20260619 12:25:23.691843 17769 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:41425 every 8 connection(s)
I20260619 12:25:23.692754 17641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:25:23.700886 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 17641
I20260619 12:25:23.701069 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:25:23.701354 7071 external_mini_cluster.cc:1428] Setting key fc68234201388478cf84e2ddf58065a5
Jun 19 12:25:23 dist-test-slave-kr9h krb5kdc[17267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871923, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:25:23.706954 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.694689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:39513 (local address 127.6.231.254:36749)
0619 12:25:23.694839 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:23.694843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:23.695820 (+ 977us) server_negotiation.cc:407] Connection header received
0619 12:25:23.697234 (+ 1414us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:23.697238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:23.697293 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:23.697405 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:23.698636 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.699172 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.700313 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.700540 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.700664 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.703501 (+ 2837us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:23.703504 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:23.703507 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:25:23.703537 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:23.705037 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:23.705625 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:23.705628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:23.705629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:23.705708 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:23.706032 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:23.706034 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:23.706035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:23.706210 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:23.706285 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:23.706736 (+ 451us) server_negotiation.cc:299] Negotiation successful
0619 12:25:23.706831 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 12:25:23.707793 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.695083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36749 (local address 127.6.231.195:39513)
0619 12:25:23.695621 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:23.695672 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:23.696989 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:23.697603 (+ 614us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:23.697609 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:23.697972 (+ 363us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:25:23.698475 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.698492 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.699296 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.699300 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.700189 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.700197 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.700714 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.700718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.700883 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.701585 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:25:23.701608 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:25:23.703337 (+ 1729us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:25:23.705142 (+ 1805us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:25:23.705170 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:25:23.705182 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:25:23.705490 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:25:23.705800 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:25:23.705802 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:25:23.705804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:25:23.705941 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:25:23.706293 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:25:23.706298 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:25:23.706522 (+ 224us) client_negotiation.cc:769] Sending connection context
0619 12:25:23.706670 (+ 148us) client_negotiation.cc:240] Negotiation successful
0619 12:25:23.706915 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":118,"threads_started":1}
I20260619 12:25:23.708997 17770 heartbeater.cc:347] Connected to a master server at 127.6.231.254:36749
I20260619 12:25:23.709257 17770 heartbeater.cc:464] Registering TS with master...
I20260619 12:25:23.709860 17770 heartbeater.cc:511] Master 127.6.231.254:36749 requested a full tablet report, sending...
I20260619 12:25:23.710948 17312 ts_manager.cc:194] Registered new tserver with Master: b378c381832f43ec8f1ce482c780329b (127.6.231.195:41425)
I20260619 12:25:23.711551 17312 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:39513
I20260619 12:25:23.715653 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:25:23.725165 17351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.717709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45986 (local address 127.6.231.254:36749)
0619 12:25:23.717835 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:23.717838 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:23.718110 (+ 272us) server_negotiation.cc:407] Connection header received
0619 12:25:23.718209 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:23.718212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:23.718264 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:23.718334 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:25:23.719023 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.719561 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.720438 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.720669 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.720824 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.722197 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:25:23.722201 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:25:23.722204 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:25:23.722229 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:25:23.723505 (+ 1276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:23.723920 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:23.723924 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:23.723926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:23.724008 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:25:23.724279 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:25:23.724284 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:25:23.724286 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:25:23.724496 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:25:23.724634 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:23.724864 (+ 230us) server_negotiation.cc:299] Negotiation successful
0619 12:25:23.724976 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 12:25:23.727938 17312 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45986:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:25:23.731112 17312 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:25:23.745962 17781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.739535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:40485 (local address 127.0.0.1:58072)
0619 12:25:23.740137 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:23.740186 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:23.740305 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:23.741309 (+ 1004us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:23.741312 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:23.741322 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:25:23.741582 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.741590 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.743026 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.743030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.744285 (+ 1255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.744295 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.745385 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.745389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.745523 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.745562 (+ 39us) client_negotiation.cc:769] Sending connection context
0619 12:25:23.745676 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 12:25:23.745801 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":224,"spinlock_wait_cycles":1920,"thread_start_us":83,"threads_started":1}
I20260619 12:25:23.745985 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.739650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58072 (local address 127.6.231.194:40485)
0619 12:25:23.740813 (+ 1163us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:23.740827 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:23.740852 (+ 25us) server_negotiation.cc:407] Connection header received
0619 12:25:23.740934 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:23.740940 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:23.741145 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:23.741515 (+ 370us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:25:23.741690 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.742899 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.744392 (+ 1493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.745237 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.745378 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.745530 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:23.745725 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 12:25:23.745821 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1061,"thread_start_us":145,"threads_started":1}
I20260619 12:25:23.746129 17784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.739983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51072 (local address 127.6.231.195:41425)
0619 12:25:23.740438 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:23.740450 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:23.740821 (+ 371us) server_negotiation.cc:407] Connection header received
0619 12:25:23.740905 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:23.740910 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:23.741115 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:23.741287 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:25:23.741705 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.743183 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.744453 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.745237 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.745355 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.745849 (+ 494us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:23.745896 (+ 47us) server_negotiation.cc:299] Negotiation successful
0619 12:25:23.745975 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":143,"threads_started":1}
I20260619 12:25:23.746461 17785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.739609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:41425 (local address 127.0.0.1:51072)
0619 12:25:23.740612 (+ 1003us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:23.740627 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:23.740722 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:23.741282 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:23.741291 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:23.741316 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:25:23.741570 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.741578 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.743362 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.743365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.744291 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.744323 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.745507 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.745511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.745610 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.745629 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 12:25:23.745715 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 12:25:23.746320 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":933,"mutex_wait_us":598,"thread_start_us":111,"threads_started":1}
I20260619 12:25:23.746840 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.740278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:34165 (local address 127.0.0.1:39410)
0619 12:25:23.740974 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:23.740987 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:23.741065 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:23.741699 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:23.741702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:23.741718 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:25:23.741798 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.741802 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.742884 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.742891 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.745040 (+ 2149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.745048 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.746445 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.746448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.746538 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.746558 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 12:25:23.746623 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 12:25:23.746689 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260619 12:25:23.747269 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.740775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39410 (local address 127.6.231.193:34165)
0619 12:25:23.741128 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:23.741142 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:23.741162 (+ 20us) server_negotiation.cc:407] Connection header received
0619 12:25:23.741237 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:23.741242 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:23.741408 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:23.741588 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:25:23.741901 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.742755 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.745151 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.746135 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.746252 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.746391 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:23.747048 (+ 657us) server_negotiation.cc:299] Negotiation successful
0619 12:25:23.747138 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":90,"threads_started":1}
I20260619 12:25:23.749913 17704 tablet_service.cc:1467] Processing CreateTablet for tablet 95a79f0536904b7cb51659fd1b75cb41 (DEFAULT_TABLE table=test-table [id=ff3cb6680dc844fb8c87285e1f5bb960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:25:23.750895 17704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95a79f0536904b7cb51659fd1b75cb41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:25:23.751044 17436 tablet_service.cc:1467] Processing CreateTablet for tablet 95a79f0536904b7cb51659fd1b75cb41 (DEFAULT_TABLE table=test-table [id=ff3cb6680dc844fb8c87285e1f5bb960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:25:23.751870 17436 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95a79f0536904b7cb51659fd1b75cb41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:25:23.755942 17570 tablet_service.cc:1467] Processing CreateTablet for tablet 95a79f0536904b7cb51659fd1b75cb41 (DEFAULT_TABLE table=test-table [id=ff3cb6680dc844fb8c87285e1f5bb960]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:25:23.756946 17570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95a79f0536904b7cb51659fd1b75cb41. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:25:23.757546 17788 tablet_bootstrap.cc:492] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9: Bootstrap starting.
I20260619 12:25:23.757936 17787 tablet_bootstrap.cc:492] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b: Bootstrap starting.
I20260619 12:25:23.760915 17787 tablet_bootstrap.cc:654] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b: Neither blocks nor log segments found. Creating new log.
I20260619 12:25:23.762311 17787 log.cc:824] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b: Log is configured to *not* fsync() on all Append() calls
I20260619 12:25:23.763233 17788 tablet_bootstrap.cc:654] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9: Neither blocks nor log segments found. Creating new log.
I20260619 12:25:23.763844 17790 tablet_bootstrap.cc:492] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b: Bootstrap starting.
I20260619 12:25:23.764441 17788 log.cc:824] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9: Log is configured to *not* fsync() on all Append() calls
I20260619 12:25:23.764485 17787 tablet_bootstrap.cc:492] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b: No bootstrap required, opened a new log
I20260619 12:25:23.764694 17787 ts_tablet_manager.cc:1397] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260619 12:25:23.766417 17788 tablet_bootstrap.cc:492] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9: No bootstrap required, opened a new log
I20260619 12:25:23.766510 17790 tablet_bootstrap.cc:654] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b: Neither blocks nor log segments found. Creating new log.
I20260619 12:25:23.766609 17788 ts_tablet_manager.cc:1397] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.000s
I20260619 12:25:23.767231 17790 log.cc:824] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b: Log is configured to *not* fsync() on all Append() calls
I20260619 12:25:23.768710 17790 tablet_bootstrap.cc:492] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b: No bootstrap required, opened a new log
I20260619 12:25:23.768903 17790 ts_tablet_manager.cc:1397] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 12:25:23.768819 17787 raft_consensus.cc:348] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac2a0553fee949769d9c96acfcf3e24b" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 40485 } } peers { permanent_uuid: "b378c381832f43ec8f1ce482c780329b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 41425 } } peers { permanent_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 34165 } }
I20260619 12:25:23.769098 17787 raft_consensus.cc:374] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:25:23.769187 17787 raft_consensus.cc:729] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b378c381832f43ec8f1ce482c780329b, State: Initialized, Role: FOLLOWER
I20260619 12:25:23.769666 17787 consensus_queue.cc:260] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac2a0553fee949769d9c96acfcf3e24b" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 40485 } } peers { permanent_uuid: "b378c381832f43ec8f1ce482c780329b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 41425 } } peers { permanent_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 34165 } }
I20260619 12:25:23.770313 17770 heartbeater.cc:503] Master 127.6.231.254:36749 was elected leader, sending a full tablet report...
I20260619 12:25:23.770642 17787 ts_tablet_manager.cc:1428] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 12:25:23.771456 17788 raft_consensus.cc:348] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac2a0553fee949769d9c96acfcf3e24b" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 40485 } } peers { permanent_uuid: "b378c381832f43ec8f1ce482c780329b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 41425 } } peers { permanent_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 34165 } }
I20260619 12:25:23.771718 17788 raft_consensus.cc:374] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:25:23.771777 17788 raft_consensus.cc:729] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 622cbdb76f3f49a58f07a5491e4dcfa9, State: Initialized, Role: FOLLOWER
I20260619 12:25:23.772217 17788 consensus_queue.cc:260] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac2a0553fee949769d9c96acfcf3e24b" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 40485 } } peers { permanent_uuid: "b378c381832f43ec8f1ce482c780329b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 41425 } } peers { permanent_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 34165 } }
I20260619 12:25:23.772763 17502 heartbeater.cc:503] Master 127.6.231.254:36749 was elected leader, sending a full tablet report...
I20260619 12:25:23.772579 17790 raft_consensus.cc:348] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac2a0553fee949769d9c96acfcf3e24b" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 40485 } } peers { permanent_uuid: "b378c381832f43ec8f1ce482c780329b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 41425 } } peers { permanent_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 34165 } }
I20260619 12:25:23.772850 17790 raft_consensus.cc:374] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:25:23.772912 17790 raft_consensus.cc:729] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac2a0553fee949769d9c96acfcf3e24b, State: Initialized, Role: FOLLOWER
I20260619 12:25:23.773322 17790 consensus_queue.cc:260] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac2a0553fee949769d9c96acfcf3e24b" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 40485 } } peers { permanent_uuid: "b378c381832f43ec8f1ce482c780329b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 41425 } } peers { permanent_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 34165 } }
I20260619 12:25:23.774354 17790 ts_tablet_manager.cc:1428] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260619 12:25:23.774706 17788 ts_tablet_manager.cc:1428] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9: Time spent starting tablet: real 0.008s user 0.002s sys 0.003s
I20260619 12:25:23.774737 17636 heartbeater.cc:503] Master 127.6.231.254:36749 was elected leader, sending a full tablet report...
I20260619 12:25:23.893810 17794 raft_consensus.cc:482] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:25:23.894061 17794 raft_consensus.cc:504] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac2a0553fee949769d9c96acfcf3e24b" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 40485 } } peers { permanent_uuid: "b378c381832f43ec8f1ce482c780329b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 41425 } } peers { permanent_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 34165 } }
I20260619 12:25:23.895334 17794 leader_election.cc:288] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ac2a0553fee949769d9c96acfcf3e24b (127.6.231.194:40485), b378c381832f43ec8f1ce482c780329b (127.6.231.195:41425)
I20260619 12:25:23.899246 17783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.895718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:39729 (local address 127.6.231.194:40485)
0619 12:25:23.895859 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:23.895866 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:23.896091 (+ 225us) server_negotiation.cc:407] Connection header received
0619 12:25:23.896264 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:23.896267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:23.896311 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:23.896397 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:25:23.896684 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.897188 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.898268 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.898680 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.898810 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.898866 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:23.899048 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 12:25:23.899113 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 12:25:23.899258 17797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.895598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:40485 (local address 127.6.231.193:39729)
0619 12:25:23.895974 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:23.895986 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:23.896137 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:23.896409 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:23.896412 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:23.896427 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:25:23.896548 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.896555 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.897299 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.897304 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.898137 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.898146 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.898803 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.898807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.898900 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.898923 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 12:25:23.899022 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 12:25:23.899105 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":124,"threads_started":1}
I20260619 12:25:23.899322 17784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.895750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:56685 (local address 127.6.231.195:41425)
0619 12:25:23.895879 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:23.895884 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:23.896159 (+ 275us) server_negotiation.cc:407] Connection header received
0619 12:25:23.896321 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:23.896326 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:23.896372 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:23.896473 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:25:23.896747 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.897245 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.898335 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.898759 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.898888 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.898941 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:23.899128 (+ 187us) server_negotiation.cc:299] Negotiation successful
0619 12:25:23.899194 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:25:23.899391 17796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.895612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:41425 (local address 127.6.231.193:56685)
0619 12:25:23.896054 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:23.896065 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:25:23.896197 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:25:23.896489 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:25:23.896491 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:25:23.896506 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:25:23.896629 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.896636 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.897356 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.897362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.898210 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:25:23.898219 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.898879 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.898882 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:25:23.898991 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.899018 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 12:25:23.899113 (+ 95us) client_negotiation.cc:240] Negotiation successful
0619 12:25:23.899179 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":102,"threads_started":1}
I20260619 12:25:23.899896 17590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95a79f0536904b7cb51659fd1b75cb41" candidate_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac2a0553fee949769d9c96acfcf3e24b" is_pre_election: true
I20260619 12:25:23.899924 17724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95a79f0536904b7cb51659fd1b75cb41" candidate_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b378c381832f43ec8f1ce482c780329b" is_pre_election: true
I20260619 12:25:23.900199 17590 raft_consensus.cc:2454] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 622cbdb76f3f49a58f07a5491e4dcfa9 in term 0.
I20260619 12:25:23.900216 17724 raft_consensus.cc:2454] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 622cbdb76f3f49a58f07a5491e4dcfa9 in term 0.
I20260619 12:25:23.900669 17390 leader_election.cc:302] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 622cbdb76f3f49a58f07a5491e4dcfa9, ac2a0553fee949769d9c96acfcf3e24b, b378c381832f43ec8f1ce482c780329b; no voters:
I20260619 12:25:23.900954 17794 raft_consensus.cc:2781] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:25:23.901057 17794 raft_consensus.cc:482] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:25:23.901103 17794 raft_consensus.cc:3037] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:23.902417 17794 raft_consensus.cc:504] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac2a0553fee949769d9c96acfcf3e24b" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 40485 } } peers { permanent_uuid: "b378c381832f43ec8f1ce482c780329b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 41425 } } peers { permanent_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 34165 } }
I20260619 12:25:23.902796 17794 leader_election.cc:288] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [CANDIDATE]: Term 1 election: Requested vote from peers ac2a0553fee949769d9c96acfcf3e24b (127.6.231.194:40485), b378c381832f43ec8f1ce482c780329b (127.6.231.195:41425)
I20260619 12:25:23.903149 17590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95a79f0536904b7cb51659fd1b75cb41" candidate_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ac2a0553fee949769d9c96acfcf3e24b"
I20260619 12:25:23.903297 17590 raft_consensus.cc:3037] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:23.903455 17724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95a79f0536904b7cb51659fd1b75cb41" candidate_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b378c381832f43ec8f1ce482c780329b"
I20260619 12:25:23.903582 17724 raft_consensus.cc:3037] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:25:23.904707 17590 raft_consensus.cc:2454] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 622cbdb76f3f49a58f07a5491e4dcfa9 in term 1.
I20260619 12:25:23.904884 17724 raft_consensus.cc:2454] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 622cbdb76f3f49a58f07a5491e4dcfa9 in term 1.
I20260619 12:25:23.905019 17390 leader_election.cc:302] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 622cbdb76f3f49a58f07a5491e4dcfa9, ac2a0553fee949769d9c96acfcf3e24b; no voters:
I20260619 12:25:23.905181 17794 raft_consensus.cc:2781] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:25:23.905413 17794 raft_consensus.cc:686] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [term 1 LEADER]: Becoming Leader. State: Replica: 622cbdb76f3f49a58f07a5491e4dcfa9, State: Running, Role: LEADER
I20260619 12:25:23.905823 17794 consensus_queue.cc:237] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac2a0553fee949769d9c96acfcf3e24b" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 40485 } } peers { permanent_uuid: "b378c381832f43ec8f1ce482c780329b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 41425 } } peers { permanent_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 34165 } }
I20260619 12:25:23.908828 17311 catalog_manager.cc:5495] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 622cbdb76f3f49a58f07a5491e4dcfa9 (127.6.231.193). New cstate: current_term: 1 leader_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ac2a0553fee949769d9c96acfcf3e24b" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 40485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b378c381832f43ec8f1ce482c780329b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 41425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "622cbdb76f3f49a58f07a5491e4dcfa9" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 34165 } health_report { overall_health: HEALTHY } } }
W20260619 12:25:23.943686 17771 tablet.cc:2369] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:25:23.948457 17786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.944752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39422 (local address 127.6.231.193:34165)
0619 12:25:23.944889 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 12:25:23.944894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:25:23.944910 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:25:23.945007 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:25:23.945010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:25:23.945064 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:25:23.945187 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:25:23.945477 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.945981 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.947006 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:25:23.947237 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:25:23.947348 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:25:23.947656 (+ 308us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:25:23.948072 (+ 416us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:25:23.948219 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 12:25:23.948255 (+ 36us) server_negotiation.cc:299] Negotiation successful
0619 12:25:23.948311 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:25:23.954413 17724 raft_consensus.cc:1261] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b [term 1 FOLLOWER]: Refusing update from remote peer 622cbdb76f3f49a58f07a5491e4dcfa9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:25:23.954413 17590 raft_consensus.cc:1261] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b [term 1 FOLLOWER]: Refusing update from remote peer 622cbdb76f3f49a58f07a5491e4dcfa9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:25:23.955199 17794 consensus_queue.cc:1035] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b378c381832f43ec8f1ce482c780329b" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 41425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:25:23.955435 17798 consensus_queue.cc:1035] T 95a79f0536904b7cb51659fd1b75cb41 P 622cbdb76f3f49a58f07a5491e4dcfa9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ac2a0553fee949769d9c96acfcf3e24b" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 40485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:25:23.964303 17803 mvcc.cc:205] Tried to move back new op lower bound from 7298547400510185472 to 7298547400320958464. Current Snapshot: MvccSnapshot[applied={T|T < 7298547400510185472}]
I20260619 12:25:23.971772 17312 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:25:23.971967 17312 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45986:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:25:23.974627 17312 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 521787f9c3a94b66b2042360f0e42aba: Sending DeleteTablet for 3 replicas of tablet 95a79f0536904b7cb51659fd1b75cb41
I20260619 12:25:23.975435 17570 tablet_service.cc:1514] Processing DeleteTablet for tablet 95a79f0536904b7cb51659fd1b75cb41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:25:23 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:58072
I20260619 12:25:23.975561 17436 tablet_service.cc:1514] Processing DeleteTablet for tablet 95a79f0536904b7cb51659fd1b75cb41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:25:23 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:39410
I20260619 12:25:23.975956 17811 tablet_replica.cc:330] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b: stopping tablet replica
I20260619 12:25:23.975989 17704 tablet_service.cc:1514] Processing DeleteTablet for tablet 95a79f0536904b7cb51659fd1b75cb41 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:25:23 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:51072
I20260619 12:25:23.976323 17811 raft_consensus.cc:2229] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:25:23.976470 17813 tablet_replica.cc:330] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b: stopping tablet replica
I20260619 12:25:23.976637 17811 raft_consensus.cc:2258] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:25:23.976763 17813 raft_consensus.cc:2229] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:25:23.977128 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 17373
I20260619 12:25:23.977418 17813 raft_consensus.cc:2258] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:25:23.979334 17811 ts_tablet_manager.cc:1905] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:25:23.980334 17813 ts_tablet_manager.cc:1905] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:25:23.982170 17811 ts_tablet_manager.cc:1918] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:25:23.982276 17811 log.cc:1196] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/95a79f0536904b7cb51659fd1b75cb41
I20260619 12:25:23.982625 17811 ts_tablet_manager.cc:1939] T 95a79f0536904b7cb51659fd1b75cb41 P ac2a0553fee949769d9c96acfcf3e24b: Deleting consensus metadata
I20260619 12:25:23.983204 17813 ts_tablet_manager.cc:1918] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:25:23.983302 17813 log.cc:1196] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/95a79f0536904b7cb51659fd1b75cb41
I20260619 12:25:23.983639 17813 ts_tablet_manager.cc:1939] T 95a79f0536904b7cb51659fd1b75cb41 P b378c381832f43ec8f1ce482c780329b: Deleting consensus metadata
I20260619 12:25:23.984123 17297 catalog_manager.cc:4841] TS ac2a0553fee949769d9c96acfcf3e24b (127.6.231.194:40485): tablet 95a79f0536904b7cb51659fd1b75cb41 (table test-table [id=ff3cb6680dc844fb8c87285e1f5bb960]) successfully deleted
I20260619 12:25:23.984812 17296 catalog_manager.cc:4841] TS b378c381832f43ec8f1ce482c780329b (127.6.231.195:41425): tablet 95a79f0536904b7cb51659fd1b75cb41 (table test-table [id=ff3cb6680dc844fb8c87285e1f5bb960]) successfully deleted
I20260619 12:25:23.988613 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 17507
W20260619 12:25:23.988611 17299 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:34165 (error 108)
I20260619 12:25:23.990907 17782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:25:23.990616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:34165 (local address 127.0.0.1:39424)
0619 12:25:23.990761 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 12:25:23.990827 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:34165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 12:25:23.991207 17299 catalog_manager.cc:4584] TS 622cbdb76f3f49a58f07a5491e4dcfa9 (127.6.231.193:34165): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95a79f0536904b7cb51659fd1b75cb41: Network error: Client connection negotiation failed: client connection to 127.6.231.193:34165: connect: Connection refused (error 111)
I20260619 12:25:23.994590 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 17641
I20260619 12:25:23.999908 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 17280
2026-06-19T12:25:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2401 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:25:24 dist-test-slave-kr9h krb5kdc[17817](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:25:24 dist-test-slave-kr9h krb5kdc[17817](info): set up 2 sockets
Jun 19 12:25:24 dist-test-slave-kr9h krb5kdc[17817](info): commencing operation
krb5kdc: starting...
W20260619 12:25:25.046370 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:25:25 dist-test-slave-kr9h krb5kdc[17817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:25:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:25:25Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.231.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.231.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/rangeradmin_127.6.231.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/rangeradmin_127.6.231.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.231.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.231.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/rangerlookup_127.6.231.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/rangerlookup_127.6.231.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.231.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.231.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/HTTP_127.6.231.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/HTTP_127.6.231.212@KRBTEST.COM.keytab.
I20260619 12:25:25.216502 7071 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskYaZBJE/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/postgres -l logfile start
2026-06-19 12:25:27.759 UTC [17847] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 12:25:27.759 UTC [17847] LOG: listening on IPv4 address "127.6.231.212", port 50863
2026-06-19 12:25:27.763 UTC [17847] LOG: listening on Unix socket "/tmp/.s.PGSQL.50863"
2026-06-19 12:25:27.785 UTC [17850] LOG: database system was shut down at 2026-06-19 12:25:26 UTC
2026-06-19 12:25:27.789 UTC [17847] LOG: database system is ready to accept connections
I20260619 12:25:28.734292 7071 mini_postgres.cc:96] Postgres bound to 50863
2026-06-19 12:25:28.741 UTC [17858] FATAL: database "slave" does not exist
127.6.231.212:50863 - accepting connections
I20260619 12:25:28.741930 7071 mini_ranger.cc:161] Starting Ranger...
I20260619 12:25:28.757639 7071 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 12:25:29.255980 7071 mini_ranger.cc:87] Created ranger Postgres database
I20260619 12:25:29.256080 7071 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-admin
2026-06-19 12:25:29,545 [I] DB FLAVOR :POSTGRES
2026-06-19 12:25:29,545 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 12:25:29,545 [I] Checking connection..
2026-06-19 12:25:29,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 12:25:29,807 [I] Checking connection passed.
2026-06-19 12:25:29,808 [I] --------- Verifying version history table ---------
2026-06-19 12:25:29,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:25:30,041 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 12:25:30,042 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:25:30,289 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 12:25:30,289 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 12:25:30,290 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 12:25:30.538 UTC [17922] WARNING: there is no transaction in progress
2026-06-19 12:25:30,544 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 12:25:30,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:25:30,774 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 12:25:30,774 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 12:25:30,774 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 12:25:31,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 12:25:31,273 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-kr9h.c.gcp-upstream.internal','N') ;"
2026-06-19 12:25:31,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 12:25:31,769 [I] Table x_portal_user does not exist in database ranger
2026-06-19 12:25:31,770 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 12:25:32,005 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 12:25:32,006 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 12:25:32,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 12:25:32,483 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 12:25:32,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 12:25:33.491 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.510 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.529 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.548 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.566 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.583 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.642 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.650 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.665 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.676 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.686 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.702 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.713 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.721 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.729 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:33.738 UTC [18030] WARNING: there is no transaction in progress
2026-06-19 12:25:34,480 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 12:25:34,480 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 12:25:34,728 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 12:25:34,729 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 12:25:34,962 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 12:25:34,962 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 12:25:35,200 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 12:25:35,200 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 12:25:35,445 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 12:25:35,717 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-kr9h.c.gcp-upstream.internal';"
2026-06-19 12:25:35,968 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 12:25:35,969 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 12:25:36,215 [I] DB_PATCHES have already been applied
I20260619 12:25:36.221282 7071 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-admin:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskYaZBJE/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 12:25:36.221359 7071 mini_ranger.cc:193] Using host: 127.6.231.212
I20260619 12:25:36.225189 7071 mini_ranger.cc:227] Ranger admin URL: http://127.6.231.212:52935
Jun 19, 2026 12:25:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 12:25:36 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 12:25:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 12:25:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 12:25:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 12:25:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 12:25:37 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 12:25:37.347499 7071 mini_ranger.cc:160] Time spent starting Ranger: real 8.606s user 0.007s sys 0.003s
Jun 19, 2026 12:25:58 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 12:25:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Jun 19, 2026 12:25:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 12:25:58 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 12:25:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 12:25:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 12:25:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 12:25:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 12:25:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 12:25:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 12:25:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 12:25:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 12:25:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 12:25:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 12:25:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 12:25:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 12:25:58 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 12:25:59 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 12:26:00.088135 7071 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.231.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.231.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/rangerkms_127.6.231.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/rangerkms_127.6.231.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.231.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.231.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.231.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/HTTP_127.6.231.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.231.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/HTTP_127.6.231.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 12:26:00 dist-test-slave-kr9h krb5kdc[17817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871960, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 12:26:00.178062 7071 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 12:26:00.188650 7071 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 12:26:01.026795 7071 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 12:26:01.026890 7071 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-kms
I20260619 12:26:01.026998 7071 mini_ranger_kms.cc:227] Using postgres at 127.6.231.212:50863
2026-06-19 12:26:01,211 [I] DB FLAVOR :POSTGRES
2026-06-19 12:26:01,211 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 12:26:01,211 [I] Checking connection
2026-06-19 12:26:01,211 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 12:26:01,462 [I] connection success
2026-06-19 12:26:01,463 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 12:26:01,463 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 12:26:01,463 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 12:26:01,705 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 12:26:01,705 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 12:26:01,706 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 12:26:01,706 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:50863/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 12:26:01,975 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 12:26:02.441246 7071 mini_ranger_kms.cc:310] Created kms service
I20260619 12:26:02.626427 7071 mini_ranger_kms.cc:326] Created kudu user
I20260619 12:26:02.670053 7071 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 12:26:02 dist-test-slave-kr9h krb5kdc[17817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871960, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:03.106127 7071 mini_ranger_kms.cc:384] Added ranger policy
I20260619 12:26:03.106285 7071 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-kms:/tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 12:26:03.106320 7071 mini_ranger_kms.cc:240] Using host: 127.6.231.212
I20260619 12:26:03.110610 7071 mini_ranger_kms.cc:276] Ranger KMS PID: 18211
I20260619 12:26:03.110761 7071 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.231.212:56719
Jun 19, 2026 12:26:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 12:26:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 12:26:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 12:26:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 12:26:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 12:26:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Jun 19 12:26:05 dist-test-slave-kr9h krb5kdc[17817](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781871965, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.231.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:05.414052 7071 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.236s user 0.000s sys 0.168s
I20260619 12:26:05.414209 7071 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 12:26:05.414333 7071 mini_ranger_kms.cc:401] 127.6.231.212:56719/kms/v1/keys
Jun 19 12:26:05 dist-test-slave-kr9h krb5kdc[17817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871965, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.231.212@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
Jun 19 12:26:06 dist-test-slave-kr9h krb5kdc[17817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871965, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.231.212@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
Jun 19 12:26:08 dist-test-slave-kr9h krb5kdc[17817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:26:08.548448 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:45171
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:45171
--ranger_config_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.231.212:56719/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:26:08.664366 18261 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:26:08.664662 18261 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:26:08.664711 18261 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:26:08.668901 18261 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:26:08.668982 18261 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:26:08.669003 18261 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:26:08.669019 18261 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:26:08.669034 18261 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:26:08.674727 18261 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.231.212:56719/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:45171
--ranger_config_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:45171
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18261
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:26:08.675834 18261 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:26:08.676890 18261 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:26:08.685842 18269 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:08.685876 18267 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:08.685956 18261 server_base.cc:1034] running on GCE node
W20260619 12:26:08.685851 18266 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:08.690104 18261 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:26:08.691316 18261 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:26:08.692520 18261 hybrid_clock.cc:648] HybridClock initialized: now 1781871968692503 us; error 36 us; skew 500 ppm
Jun 19 12:26:08 dist-test-slave-kr9h krb5kdc[17817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871968, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:08.698760 18261 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:26:08.699788 18261 webserver.cc:460] Webserver started at http://127.6.231.254:40065/ using document root <none> and password file <none>
I20260619 12:26:08.700348 18261 fs_manager.cc:359] Metadata directory not provided
I20260619 12:26:08.700426 18261 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:26:08.700647 18261 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:26:08 dist-test-slave-kr9h krb5kdc[17817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871968, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:08.764448 18261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "2b38a86d0cf84837a85fe5fa547f3c9d"
format_stamp: "Formatted at 2026-06-19 12:26:08 on dist-test-slave-kr9h"
server_key: "8978764d9aba4b772b01c9e61f4faa36"
server_key_iv: "41a3e99e1fd7df7de2d8ee79362c3ebb"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:08.765094 18261 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "2b38a86d0cf84837a85fe5fa547f3c9d"
format_stamp: "Formatted at 2026-06-19 12:26:08 on dist-test-slave-kr9h"
server_key: "8978764d9aba4b772b01c9e61f4faa36"
server_key_iv: "41a3e99e1fd7df7de2d8ee79362c3ebb"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:08.769138 18261 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260619 12:26:08.786528 18276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:08.788022 18261 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 12:26:08.788156 18261 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "2b38a86d0cf84837a85fe5fa547f3c9d"
format_stamp: "Formatted at 2026-06-19 12:26:08 on dist-test-slave-kr9h"
server_key: "8978764d9aba4b772b01c9e61f4faa36"
server_key_iv: "41a3e99e1fd7df7de2d8ee79362c3ebb"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:08.788291 18261 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:26:08.802661 18261 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:26:08.805794 18261 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:26:08.805998 18261 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:26:08.814275 18261 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:45171
I20260619 12:26:08.814335 18328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:45171 every 8 connection(s)
I20260619 12:26:08.815280 18261 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:26:08.816780 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 18261
I20260619 12:26:08.816989 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:26:08.817291 7071 external_mini_cluster.cc:1428] Setting key a3525c67b090615d012be3cc3565801c
I20260619 12:26:08.818825 18329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:26:08.826503 18329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d: Bootstrap starting.
Jun 19 12:26:08 dist-test-slave-kr9h krb5kdc[17817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:26:08.829218 18329 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d: Neither blocks nor log segments found. Creating new log.
I20260619 12:26:08.830185 18329 log.cc:824] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d: Log is configured to *not* fsync() on all Append() calls
I20260619 12:26:08.832656 18329 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d: No bootstrap required, opened a new log
I20260619 12:26:08.835611 18332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:08.818666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39800 (local address 127.6.231.254:45171)
0619 12:26:08.819110 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:08.819125 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:08.821772 (+ 2647us) server_negotiation.cc:407] Connection header received
0619 12:26:08.822672 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:08.822690 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:08.823045 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:08.823401 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:26:08.824259 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:08.825317 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:08.826259 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:08.826837 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:08.827020 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:08.829849 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:26:08.829854 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:26:08.829868 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:26:08.829896 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:26:08.832477 (+ 2581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:08.833045 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:08.833049 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:08.833054 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:08.833159 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:08.833495 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:08.833498 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:08.833499 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:08.833899 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:26:08.834111 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:08.834448 (+ 337us) server_negotiation.cc:299] Negotiation successful
0619 12:26:08.834856 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":149,"threads_started":1}
I20260619 12:26:08.836658 18329 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b38a86d0cf84837a85fe5fa547f3c9d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45171 } }
I20260619 12:26:08.836948 18329 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:26:08.837038 18329 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b38a86d0cf84837a85fe5fa547f3c9d, State: Initialized, Role: FOLLOWER
I20260619 12:26:08.837558 18329 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b38a86d0cf84837a85fe5fa547f3c9d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45171 } }
I20260619 12:26:08.837711 18329 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:26:08.837786 18329 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:26:08.837881 18329 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:26:08.838790 18329 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b38a86d0cf84837a85fe5fa547f3c9d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45171 } }
I20260619 12:26:08.839119 18329 leader_election.cc:302] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b38a86d0cf84837a85fe5fa547f3c9d; no voters:
I20260619 12:26:08.839468 18329 leader_election.cc:288] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:26:08.839565 18334 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:26:08.839781 18334 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [term 1 LEADER]: Becoming Leader. State: Replica: 2b38a86d0cf84837a85fe5fa547f3c9d, State: Running, Role: LEADER
I20260619 12:26:08.840142 18334 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b38a86d0cf84837a85fe5fa547f3c9d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45171 } }
I20260619 12:26:08.840682 18329 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:26:08.841977 18335 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b38a86d0cf84837a85fe5fa547f3c9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b38a86d0cf84837a85fe5fa547f3c9d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45171 } } }
I20260619 12:26:08.842010 18336 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b38a86d0cf84837a85fe5fa547f3c9d. Latest consensus state: current_term: 1 leader_uuid: "2b38a86d0cf84837a85fe5fa547f3c9d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b38a86d0cf84837a85fe5fa547f3c9d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45171 } } }
I20260619 12:26:08.842097 18335 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [sys.catalog]: This master's current role is: LEADER
I20260619 12:26:08.842099 18336 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d [sys.catalog]: This master's current role is: LEADER
I20260619 12:26:08.843113 18344 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:26:08.843441 18329 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 12:26:08.849037 18344 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:26:08.857160 18344 catalog_manager.cc:1347] Generated new cluster ID: d4c3bc69618f43b287f7db18a28d7599
I20260619 12:26:08.857228 18344 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:26:08.867308 18344 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:26:08.868597 18344 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:26:08.876730 18344 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d: Generated new TSK 0
I20260619 12:26:08.877592 18344 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:26:09.573021 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.231.212:56719/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45171
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:26:09.740810 18374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:26:09.741209 18374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:26:09.741285 18374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:26:09.747584 18374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:26:09.747669 18374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:26:09.747772 18374 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:26:09.756218 18374 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.231.212:56719/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:26:09.757725 18374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:26:09.759207 18374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:26:09.771816 18384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:09.771826 18382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:09.772236 18381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:09.772609 18374 server_base.cc:1034] running on GCE node
I20260619 12:26:09.773093 18374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:26:09.773955 18374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:26:09.775159 18374 hybrid_clock.cc:648] HybridClock initialized: now 1781871969775132 us; error 42 us; skew 500 ppm
Jun 19 12:26:09 dist-test-slave-kr9h krb5kdc[17817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:09.778827 18374 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:26:09.779883 18374 webserver.cc:460] Webserver started at http://127.6.231.193:35613/ using document root <none> and password file <none>
I20260619 12:26:09.780593 18374 fs_manager.cc:359] Metadata directory not provided
I20260619 12:26:09.780689 18374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:26:09.780916 18374 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:26:09 dist-test-slave-kr9h krb5kdc[17817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:09.798866 18374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "1e50c7150c0c4355840dcdb446ebabed"
format_stamp: "Formatted at 2026-06-19 12:26:09 on dist-test-slave-kr9h"
server_key: "c0c64d8005f9f2263b2881ce25d4f2fe"
server_key_iv: "87f92d3eee8bba40103615f5c2880bc3"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:09.799587 18374 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "1e50c7150c0c4355840dcdb446ebabed"
format_stamp: "Formatted at 2026-06-19 12:26:09 on dist-test-slave-kr9h"
server_key: "c0c64d8005f9f2263b2881ce25d4f2fe"
server_key_iv: "87f92d3eee8bba40103615f5c2880bc3"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:09.807188 18374 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.004s sys 0.000s
I20260619 12:26:09.834422 18391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:09.836145 18374 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:26:09.836377 18374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "1e50c7150c0c4355840dcdb446ebabed"
format_stamp: "Formatted at 2026-06-19 12:26:09 on dist-test-slave-kr9h"
server_key: "c0c64d8005f9f2263b2881ce25d4f2fe"
server_key_iv: "87f92d3eee8bba40103615f5c2880bc3"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:09.836572 18374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:26:09.872373 18374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:26:09.878523 18374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:26:09.878806 18374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:26:09.879637 18374 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:26:09.880743 18374 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:26:09.880861 18374 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:09.880988 18374 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:26:09.881069 18374 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:09.895430 18374 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:40323
I20260619 12:26:09.896700 18374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:26:09.897676 18506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:40323 every 8 connection(s)
I20260619 12:26:09.905475 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 18374
I20260619 12:26:09.905616 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:26:09.905922 7071 external_mini_cluster.cc:1428] Setting key eaec67aa2fd3d80c1102abe40ffed8d4
Jun 19 12:26:09 dist-test-slave-kr9h krb5kdc[17817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:26:09.917865 18510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:09.901531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:45503 (local address 127.6.231.254:45171)
0619 12:26:09.903700 (+ 2169us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:09.903704 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:09.903720 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:26:09.903764 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:09.903767 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:09.903816 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:09.903903 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:26:09.905462 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:09.906316 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:09.907992 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:09.908363 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:09.908448 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:09.912124 (+ 3676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:26:09.912128 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:26:09.912131 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:26:09.912161 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:26:09.914228 (+ 2067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:09.914925 (+ 697us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:09.914929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:09.914931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:09.915027 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:09.915469 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:09.915473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:09.915474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:09.915708 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:26:09.915809 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:09.917550 (+ 1741us) server_negotiation.cc:299] Negotiation successful
0619 12:26:09.917684 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2084,"thread_start_us":89,"threads_started":1}
I20260619 12:26:09.918648 18509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:09.900415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45171 (local address 127.6.231.193:45503)
0619 12:26:09.902146 (+ 1731us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:09.902202 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:09.903272 (+ 1070us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:09.904200 (+ 928us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:09.904209 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:09.904576 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:26:09.905266 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:09.905287 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:09.906471 (+ 1184us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:09.906478 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:09.907848 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:09.907859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:09.908668 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:09.908672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:09.908879 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:09.909589 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:26:09.909610 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:26:09.911918 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:26:09.914376 (+ 2458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:09.914382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:09.914401 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:09.914796 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:09.915139 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:09.915142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:09.915144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:09.915348 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:09.915865 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:26:09.915872 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:26:09.916204 (+ 332us) client_negotiation.cc:769] Sending connection context
0619 12:26:09.916446 (+ 242us) client_negotiation.cc:240] Negotiation successful
0619 12:26:09.916869 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1503,"thread_start_us":132,"threads_started":1}
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
I20260619 12:26:09.920248 18507 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45171
I20260619 12:26:09.920507 18507 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:09.921120 18507 heartbeater.cc:511] Master 127.6.231.254:45171 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:26:09.923031 18293 ts_manager.cc:194] Registered new tserver with Master: 1e50c7150c0c4355840dcdb446ebabed (127.6.231.193:40323)
I20260619 12:26:09.924594 18293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:45503
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Jun 19 12:26:09 dist-test-slave-kr9h krb5kdc[17817](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781871969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:09.942703 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.231.212:56719/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45171
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:26:10.072768 18513 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:26:10.073069 18513 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:26:10.073186 18513 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:26:10.077718 18513 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:26:10.077800 18513 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:26:10.077900 18513 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:26:10.083292 18513 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.231.212:56719/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18513
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:26:10.084434 18513 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:26:10.085559 18513 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:26:10.097545 18520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:10.097862 18513 server_base.cc:1034] running on GCE node
W20260619 12:26:10.097944 18522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:10.098114 18519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:10.098642 18513 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:26:10.099500 18513 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:26:10.100706 18513 hybrid_clock.cc:648] HybridClock initialized: now 1781871970100685 us; error 30 us; skew 500 ppm
Jun 19 12:26:10 dist-test-slave-kr9h krb5kdc[17817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871970, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:10.104580 18513 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:26:10.105820 18513 webserver.cc:460] Webserver started at http://127.6.231.194:36379/ using document root <none> and password file <none>
I20260619 12:26:10.106580 18513 fs_manager.cc:359] Metadata directory not provided
I20260619 12:26:10.106703 18513 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:26:10.106988 18513 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:26:10 dist-test-slave-kr9h krb5kdc[17817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871970, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:10.125106 18513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "baa7bb5518cf42c2b0e963a724c5a348"
format_stamp: "Formatted at 2026-06-19 12:26:10 on dist-test-slave-kr9h"
server_key: "ea94926b46d213de148bc8c8d24c42b2"
server_key_iv: "48051ed77db013af898aa17699d03f16"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:10.126988 18513 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "baa7bb5518cf42c2b0e963a724c5a348"
format_stamp: "Formatted at 2026-06-19 12:26:10 on dist-test-slave-kr9h"
server_key: "ea94926b46d213de148bc8c8d24c42b2"
server_key_iv: "48051ed77db013af898aa17699d03f16"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:10.138442 18513 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.004s sys 0.000s
I20260619 12:26:10.160537 18529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:10.162423 18513 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260619 12:26:10.162621 18513 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "baa7bb5518cf42c2b0e963a724c5a348"
format_stamp: "Formatted at 2026-06-19 12:26:10 on dist-test-slave-kr9h"
server_key: "ea94926b46d213de148bc8c8d24c42b2"
server_key_iv: "48051ed77db013af898aa17699d03f16"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:10.162870 18513 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:26:10.191485 18513 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:26:10.195420 18513 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:26:10.195698 18513 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:26:10.196506 18513 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:26:10.197655 18513 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:26:10.197875 18513 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:10.197989 18513 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:26:10.198060 18513 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:10.211997 18513 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:44865
I20260619 12:26:10.213375 18513 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:26:10.214461 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 18513
I20260619 12:26:10.214593 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:26:10.214944 7071 external_mini_cluster.cc:1428] Setting key c0beb8416cf839f43ea1e2e2f8666898
I20260619 12:26:10.215378 18642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:44865 every 8 connection(s)
Jun 19 12:26:10 dist-test-slave-kr9h krb5kdc[17817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871970, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:26:10.241974 18510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.218772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:36341 (local address 127.6.231.254:45171)
0619 12:26:10.218898 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:10.218902 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:10.219781 (+ 879us) server_negotiation.cc:407] Connection header received
0619 12:26:10.221026 (+ 1245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:10.221031 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:10.221084 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:10.221138 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:26:10.222784 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.223641 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.228302 (+ 4661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.228700 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.228799 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.233591 (+ 4792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:26:10.233596 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:26:10.233599 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:26:10.233630 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:26:10.235592 (+ 1962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:10.236294 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:10.236298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:10.236299 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:10.236389 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:10.236821 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:10.236824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:10.236826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:10.237055 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:26:10.237137 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:10.241598 (+ 4461us) server_negotiation.cc:299] Negotiation successful
0619 12:26:10.241770 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:26:10.244004 18645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.217665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45171 (local address 127.6.231.194:36341)
0619 12:26:10.219615 (+ 1950us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:10.219670 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:10.220767 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:10.221588 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:10.221598 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:10.221963 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:26:10.222604 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.222621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.223812 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.223816 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.225201 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.225211 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.229027 (+ 3816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.229031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.229245 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.230196 (+ 951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:26:10.230217 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:26:10.231968 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:26:10.235766 (+ 3798us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:10.235774 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:10.235795 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:10.236167 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:10.236498 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:10.236502 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:10.236504 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:10.236694 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:10.237204 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:26:10.237212 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:26:10.237598 (+ 386us) client_negotiation.cc:769] Sending connection context
0619 12:26:10.242463 (+ 4865us) client_negotiation.cc:240] Negotiation successful
0619 12:26:10.242871 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1735,"thread_start_us":148,"threads_started":1}
I20260619 12:26:10.245884 18643 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45171
I20260619 12:26:10.246134 18643 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:10.246711 18643 heartbeater.cc:511] Master 127.6.231.254:45171 requested a full tablet report, sending...
I20260619 12:26:10.247855 18293 ts_manager.cc:194] Registered new tserver with Master: baa7bb5518cf42c2b0e963a724c5a348 (127.6.231.194:44865)
I20260619 12:26:10.248975 18293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:36341
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:26:10.272960 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.231.212:56719/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45171
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:34545
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:26:10.448711 18648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:26:10.449575 18648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:26:10.449725 18648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:26:10.456301 18648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:26:10.456506 18648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:26:10.456702 18648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:26:10.464834 18648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:34545
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.231.212:56719/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45171
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:26:10.466665 18648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:26:10.468135 18648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:26:10.481411 18653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:10.481624 18656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:10.484213 18648 server_base.cc:1034] running on GCE node
W20260619 12:26:10.484391 18654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:10.485116 18648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:26:10.486176 18648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:26:10.494161 18648 hybrid_clock.cc:648] HybridClock initialized: now 1781871970494066 us; error 110 us; skew 500 ppm
Jun 19 12:26:10 dist-test-slave-kr9h krb5kdc[17817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871970, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:10.498056 18648 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:26:10.499279 18648 webserver.cc:460] Webserver started at http://127.6.231.195:36141/ using document root <none> and password file <none>
I20260619 12:26:10.500000 18648 fs_manager.cc:359] Metadata directory not provided
I20260619 12:26:10.500100 18648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:26:10.500358 18648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:26:10 dist-test-slave-kr9h krb5kdc[17817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871970, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
Jun 19 12:26:10 dist-test-slave-kr9h krb5kdc[17817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871969, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:10.524919 18648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "e234e3952e454297bab3e40915c753c8"
format_stamp: "Formatted at 2026-06-19 12:26:10 on dist-test-slave-kr9h"
server_key: "717394a87531fee41512e81eeeb60ebe"
server_key_iv: "dbc2d0310af980aba1fde70904405a43"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:10.525691 18648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "e234e3952e454297bab3e40915c753c8"
format_stamp: "Formatted at 2026-06-19 12:26:10 on dist-test-slave-kr9h"
server_key: "717394a87531fee41512e81eeeb60ebe"
server_key_iv: "dbc2d0310af980aba1fde70904405a43"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:10.532416 18648 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.000s sys 0.004s
I20260619 12:26:10.559885 18663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:10.565111 18648 fs_manager.cc:725] Time spent opening block manager: real 0.014s user 0.003s sys 0.000s
I20260619 12:26:10.565307 18648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "e234e3952e454297bab3e40915c753c8"
format_stamp: "Formatted at 2026-06-19 12:26:10 on dist-test-slave-kr9h"
server_key: "717394a87531fee41512e81eeeb60ebe"
server_key_iv: "dbc2d0310af980aba1fde70904405a43"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:10.565490 18648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:26:10.588513 18648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:26:10.592948 18648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:26:10.593225 18648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:26:10.593982 18648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:26:10.595079 18648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:26:10.595183 18648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:10.595271 18648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:26:10.595335 18648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:10.608076 18648 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:42083
I20260619 12:26:10.609345 18648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:26:10.610281 18776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:42083 every 8 connection(s)
I20260619 12:26:10.617874 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 18648
I20260619 12:26:10.618014 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:26:10.618331 7071 external_mini_cluster.cc:1428] Setting key 5b59be825f1bd4ce3f38c234c49c2494
Jun 19 12:26:10 dist-test-slave-kr9h krb5kdc[17817](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871970, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:26:10.636799 18510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.612598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:53425 (local address 127.6.231.254:45171)
0619 12:26:10.612761 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:10.612766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:10.621276 (+ 8510us) server_negotiation.cc:407] Connection header received
0619 12:26:10.622657 (+ 1381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:10.622660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:10.622717 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:10.622770 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:26:10.624640 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.625490 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.627239 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.627576 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.627666 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.632485 (+ 4819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:26:10.632488 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:26:10.632492 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:26:10.632522 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:26:10.634281 (+ 1759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:10.634962 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:10.634966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:10.634968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:10.635048 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:10.635448 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:10.635451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:10.635453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:10.635718 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:26:10.635792 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:10.636499 (+ 707us) server_negotiation.cc:299] Negotiation successful
0619 12:26:10.636634 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 12:26:10.638751 18779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.612883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45171 (local address 127.6.231.195:53425)
0619 12:26:10.621097 (+ 8214us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:10.621178 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:10.622378 (+ 1200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:10.623269 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:10.623281 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:10.623703 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:26:10.624452 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.624472 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.625649 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.625653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.627097 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.627108 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.627884 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.627887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.628096 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.628850 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:26:10.628871 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:26:10.630673 (+ 1802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:26:10.634429 (+ 3756us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:10.634436 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:10.634455 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:10.634849 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:10.635150 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:10.635153 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:10.635155 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:10.635348 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:10.635872 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:26:10.635879 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:26:10.636218 (+ 339us) client_negotiation.cc:769] Sending connection context
0619 12:26:10.637240 (+ 1022us) client_negotiation.cc:240] Negotiation successful
0619 12:26:10.637658 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7909,"thread_start_us":126,"threads_started":1}
I20260619 12:26:10.640563 18777 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45171
I20260619 12:26:10.640849 18777 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:10.641465 18777 heartbeater.cc:511] Master 127.6.231.254:45171 requested a full tablet report, sending...
I20260619 12:26:10.643023 18293 ts_manager.cc:194] Registered new tserver with Master: e234e3952e454297bab3e40915c753c8 (127.6.231.195:42083)
I20260619 12:26:10.643826 18293 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:53425
I20260619 12:26:10.645939 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:26:10.663973 18510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.654003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36086 (local address 127.6.231.254:45171)
0619 12:26:10.654148 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:10.654153 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:10.654569 (+ 416us) server_negotiation.cc:407] Connection header received
0619 12:26:10.654788 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:10.654792 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:10.654842 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:10.654894 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:26:10.655892 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.656679 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.657931 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.658261 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.658332 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.660369 (+ 2037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:26:10.660373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:26:10.660376 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:26:10.660404 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:26:10.662146 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:10.662647 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:10.662650 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:10.662652 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:10.662734 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:10.663017 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:10.663020 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:10.663022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:10.663244 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:26:10.663345 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:10.663679 (+ 334us) server_negotiation.cc:299] Negotiation successful
0619 12:26:10.663803 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 12:26:10.668054 18293 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:26:10.672529 18293 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:26:10.703419 18792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.686635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:44865 (local address 127.0.0.1:49172)
0619 12:26:10.690155 (+ 3520us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:10.690169 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:10.690242 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:10.691920 (+ 1678us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:10.691924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:10.691941 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:26:10.692027 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.692032 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.697192 (+ 5160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.697195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.698996 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.699006 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.702884 (+ 3878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.702890 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.702992 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.703019 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 12:26:10.703084 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 12:26:10.703227 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3452,"thread_start_us":90,"threads_started":1}
I20260619 12:26:10.704042 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.687031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49172 (local address 127.6.231.194:44865)
0619 12:26:10.687415 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:10.687431 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:10.690339 (+ 2908us) server_negotiation.cc:407] Connection header received
0619 12:26:10.690418 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:10.690424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:10.691432 (+ 1008us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:10.691575 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:26:10.692185 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.693568 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.699178 (+ 5610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.700191 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.700313 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.700486 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:10.703790 (+ 3304us) server_negotiation.cc:299] Negotiation successful
0619 12:26:10.703906 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":72,"threads_started":1}
I20260619 12:26:10.707911 18576 tablet_service.cc:1467] Processing CreateTablet for tablet 6a387ac46b64401284412134e9216e5b (DEFAULT_TABLE table=test-table [id=6387c899d0f940fc89798f9033927b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:26:10.709028 18576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a387ac46b64401284412134e9216e5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:26:10.713325 18794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.695270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:42083 (local address 127.0.0.1:45448)
0619 12:26:10.701610 (+ 6340us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:10.701626 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:10.701727 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:10.702713 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:10.702719 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:10.702737 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:26:10.702841 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.702850 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.704189 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.704192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.705302 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.705311 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.712864 (+ 7553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.712867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.712981 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.713008 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 12:26:10.713085 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 12:26:10.713177 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6264,"thread_start_us":93,"threads_started":1}
I20260619 12:26:10.714166 18795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.695708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45448 (local address 127.6.231.195:42083)
0619 12:26:10.702004 (+ 6296us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:10.702018 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:10.702045 (+ 27us) server_negotiation.cc:407] Connection header received
0619 12:26:10.702128 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:10.702134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:10.702383 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:10.702546 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:26:10.702971 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.704051 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.711668 (+ 7617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.712505 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.712623 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.712787 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:10.713875 (+ 1088us) server_negotiation.cc:299] Negotiation successful
0619 12:26:10.713989 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6180,"thread_start_us":96,"threads_started":1}
I20260619 12:26:10.704615 18790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.684555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:40323 (local address 127.0.0.1:38534)
0619 12:26:10.688724 (+ 4169us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:10.688786 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:10.688912 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:10.689784 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:10.689790 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:10.689813 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:26:10.689947 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.689954 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.695634 (+ 5680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.695637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.697050 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.697061 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.701784 (+ 4723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.701787 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.701916 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.701957 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 12:26:10.702072 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 12:26:10.704483 (+ 2411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3974,"thread_start_us":101,"threads_started":1}
I20260619 12:26:10.717389 18711 tablet_service.cc:1467] Processing CreateTablet for tablet 6a387ac46b64401284412134e9216e5b (DEFAULT_TABLE table=test-table [id=6387c899d0f940fc89798f9033927b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:26:10.718056 18791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.684716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38534 (local address 127.6.231.193:40323)
0619 12:26:10.689036 (+ 4320us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:10.689048 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:10.689073 (+ 25us) server_negotiation.cc:407] Connection header received
0619 12:26:10.689161 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:10.689166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:10.689392 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:10.689602 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:26:10.691724 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.695492 (+ 3768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.700566 (+ 5074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.701424 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.701564 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.701720 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:10.717781 (+ 16061us) server_negotiation.cc:299] Negotiation successful
0619 12:26:10.717915 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2885,"thread_start_us":107,"threads_started":1}
I20260619 12:26:10.718541 18711 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a387ac46b64401284412134e9216e5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:26:10.720831 18440 tablet_service.cc:1467] Processing CreateTablet for tablet 6a387ac46b64401284412134e9216e5b (DEFAULT_TABLE table=test-table [id=6387c899d0f940fc89798f9033927b73]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:26:10.721601 18796 tablet_bootstrap.cc:492] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348: Bootstrap starting.
I20260619 12:26:10.722057 18440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a387ac46b64401284412134e9216e5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:26:10.723986 18796 tablet_bootstrap.cc:654] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348: Neither blocks nor log segments found. Creating new log.
I20260619 12:26:10.726311 18796 log.cc:824] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348: Log is configured to *not* fsync() on all Append() calls
I20260619 12:26:10.728436 18796 tablet_bootstrap.cc:492] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348: No bootstrap required, opened a new log
I20260619 12:26:10.728627 18796 ts_tablet_manager.cc:1397] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260619 12:26:10.732532 18796 raft_consensus.cc:348] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e50c7150c0c4355840dcdb446ebabed" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 40323 } } peers { permanent_uuid: "baa7bb5518cf42c2b0e963a724c5a348" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 44865 } } peers { permanent_uuid: "e234e3952e454297bab3e40915c753c8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42083 } }
I20260619 12:26:10.732808 18796 raft_consensus.cc:374] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:26:10.732888 18796 raft_consensus.cc:729] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: baa7bb5518cf42c2b0e963a724c5a348, State: Initialized, Role: FOLLOWER
I20260619 12:26:10.733649 18796 consensus_queue.cc:260] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e50c7150c0c4355840dcdb446ebabed" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 40323 } } peers { permanent_uuid: "baa7bb5518cf42c2b0e963a724c5a348" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 44865 } } peers { permanent_uuid: "e234e3952e454297bab3e40915c753c8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42083 } }
I20260619 12:26:10.734849 18796 ts_tablet_manager.cc:1428] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260619 12:26:10.736500 18798 tablet_bootstrap.cc:492] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed: Bootstrap starting.
I20260619 12:26:10.736842 18643 heartbeater.cc:503] Master 127.6.231.254:45171 was elected leader, sending a full tablet report...
I20260619 12:26:10.738127 18800 tablet_bootstrap.cc:492] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8: Bootstrap starting.
I20260619 12:26:10.739033 18798 tablet_bootstrap.cc:654] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed: Neither blocks nor log segments found. Creating new log.
I20260619 12:26:10.739959 18798 log.cc:824] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed: Log is configured to *not* fsync() on all Append() calls
I20260619 12:26:10.741422 18800 tablet_bootstrap.cc:654] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8: Neither blocks nor log segments found. Creating new log.
I20260619 12:26:10.742529 18800 log.cc:824] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8: Log is configured to *not* fsync() on all Append() calls
I20260619 12:26:10.744179 18798 tablet_bootstrap.cc:492] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed: No bootstrap required, opened a new log
I20260619 12:26:10.744431 18798 ts_tablet_manager.cc:1397] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260619 12:26:10.744524 18800 tablet_bootstrap.cc:492] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8: No bootstrap required, opened a new log
I20260619 12:26:10.744707 18800 ts_tablet_manager.cc:1397] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260619 12:26:10.748746 18800 raft_consensus.cc:348] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e50c7150c0c4355840dcdb446ebabed" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 40323 } } peers { permanent_uuid: "baa7bb5518cf42c2b0e963a724c5a348" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 44865 } } peers { permanent_uuid: "e234e3952e454297bab3e40915c753c8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42083 } }
I20260619 12:26:10.749102 18800 raft_consensus.cc:374] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:26:10.749199 18800 raft_consensus.cc:729] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e234e3952e454297bab3e40915c753c8, State: Initialized, Role: FOLLOWER
I20260619 12:26:10.749796 18800 consensus_queue.cc:260] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e50c7150c0c4355840dcdb446ebabed" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 40323 } } peers { permanent_uuid: "baa7bb5518cf42c2b0e963a724c5a348" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 44865 } } peers { permanent_uuid: "e234e3952e454297bab3e40915c753c8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42083 } }
I20260619 12:26:10.749959 18798 raft_consensus.cc:348] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e50c7150c0c4355840dcdb446ebabed" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 40323 } } peers { permanent_uuid: "baa7bb5518cf42c2b0e963a724c5a348" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 44865 } } peers { permanent_uuid: "e234e3952e454297bab3e40915c753c8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42083 } }
I20260619 12:26:10.750265 18798 raft_consensus.cc:374] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:26:10.750360 18798 raft_consensus.cc:729] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e50c7150c0c4355840dcdb446ebabed, State: Initialized, Role: FOLLOWER
I20260619 12:26:10.750831 18798 consensus_queue.cc:260] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e50c7150c0c4355840dcdb446ebabed" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 40323 } } peers { permanent_uuid: "baa7bb5518cf42c2b0e963a724c5a348" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 44865 } } peers { permanent_uuid: "e234e3952e454297bab3e40915c753c8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42083 } }
I20260619 12:26:10.750922 18800 ts_tablet_manager.cc:1428] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260619 12:26:10.751266 18777 heartbeater.cc:503] Master 127.6.231.254:45171 was elected leader, sending a full tablet report...
I20260619 12:26:10.751855 18798 ts_tablet_manager.cc:1428] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed: Time spent starting tablet: real 0.007s user 0.005s sys 0.000s
I20260619 12:26:10.752166 18507 heartbeater.cc:503] Master 127.6.231.254:45171 was elected leader, sending a full tablet report...
I20260619 12:26:10.762183 18804 raft_consensus.cc:482] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:26:10.762392 18804 raft_consensus.cc:504] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e50c7150c0c4355840dcdb446ebabed" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 40323 } } peers { permanent_uuid: "baa7bb5518cf42c2b0e963a724c5a348" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 44865 } } peers { permanent_uuid: "e234e3952e454297bab3e40915c753c8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42083 } }
I20260619 12:26:10.763778 18804 leader_election.cc:288] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers baa7bb5518cf42c2b0e963a724c5a348 (127.6.231.194:44865), e234e3952e454297bab3e40915c753c8 (127.6.231.195:42083)
I20260619 12:26:10.776036 18793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.765543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:56689 (local address 127.6.231.194:44865)
0619 12:26:10.765796 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:10.765801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:10.766788 (+ 987us) server_negotiation.cc:407] Connection header received
0619 12:26:10.767000 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:10.767004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:10.767054 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:10.767104 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:26:10.767900 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.768698 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.773526 (+ 4828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.774143 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.774270 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.774328 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:10.775812 (+ 1484us) server_negotiation.cc:299] Negotiation successful
0619 12:26:10.775897 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260619 12:26:10.776714 18806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.764394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:44865 (local address 127.6.231.193:56689)
0619 12:26:10.766700 (+ 2306us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:10.766715 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:10.766865 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:10.767277 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:10.767280 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:10.767298 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:26:10.767439 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.767446 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.768980 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.768984 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.770350 (+ 1366us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.770361 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.774443 (+ 4082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.774447 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.774544 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.774571 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 12:26:10.776462 (+ 1891us) client_negotiation.cc:240] Negotiation successful
0619 12:26:10.776541 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2234,"thread_start_us":110,"threads_started":1}
I20260619 12:26:10.777231 18795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.765667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:36753 (local address 127.6.231.195:42083)
0619 12:26:10.765879 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:10.765883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:10.767822 (+ 1939us) server_negotiation.cc:407] Connection header received
0619 12:26:10.770626 (+ 2804us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:10.770630 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:10.770682 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:10.770735 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:26:10.771142 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.771887 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.774720 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.775320 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.775412 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.775458 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:10.777021 (+ 1563us) server_negotiation.cc:299] Negotiation successful
0619 12:26:10.777103 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260619 12:26:10.777807 18805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.764789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:42083 (local address 127.6.231.193:36753)
0619 12:26:10.767732 (+ 2943us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:10.767747 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:10.770490 (+ 2743us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:10.770873 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:10.770875 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:10.770892 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:26:10.770992 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.770998 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.772155 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.772159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.773340 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:10.773351 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.775556 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.775560 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:10.775657 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.775678 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 12:26:10.777596 (+ 1918us) client_negotiation.cc:240] Negotiation successful
0619 12:26:10.777673 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2830,"thread_start_us":93,"threads_started":1}
I20260619 12:26:10.778479 18581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a387ac46b64401284412134e9216e5b" candidate_uuid: "1e50c7150c0c4355840dcdb446ebabed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa7bb5518cf42c2b0e963a724c5a348" is_pre_election: true
I20260619 12:26:10.778823 18581 raft_consensus.cc:2454] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e50c7150c0c4355840dcdb446ebabed in term 0.
I20260619 12:26:10.778968 18727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a387ac46b64401284412134e9216e5b" candidate_uuid: "1e50c7150c0c4355840dcdb446ebabed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e234e3952e454297bab3e40915c753c8" is_pre_election: true
I20260619 12:26:10.779266 18727 raft_consensus.cc:2454] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1e50c7150c0c4355840dcdb446ebabed in term 0.
I20260619 12:26:10.779338 18395 leader_election.cc:302] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e50c7150c0c4355840dcdb446ebabed, baa7bb5518cf42c2b0e963a724c5a348; no voters:
I20260619 12:26:10.779645 18804 raft_consensus.cc:2781] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:26:10.779726 18804 raft_consensus.cc:482] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:26:10.779764 18804 raft_consensus.cc:3037] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:26:10.781090 18804 raft_consensus.cc:504] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e50c7150c0c4355840dcdb446ebabed" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 40323 } } peers { permanent_uuid: "baa7bb5518cf42c2b0e963a724c5a348" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 44865 } } peers { permanent_uuid: "e234e3952e454297bab3e40915c753c8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42083 } }
I20260619 12:26:10.781464 18804 leader_election.cc:288] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [CANDIDATE]: Term 1 election: Requested vote from peers baa7bb5518cf42c2b0e963a724c5a348 (127.6.231.194:44865), e234e3952e454297bab3e40915c753c8 (127.6.231.195:42083)
I20260619 12:26:10.781847 18727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a387ac46b64401284412134e9216e5b" candidate_uuid: "1e50c7150c0c4355840dcdb446ebabed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e234e3952e454297bab3e40915c753c8"
I20260619 12:26:10.781951 18727 raft_consensus.cc:3037] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:26:10.783166 18727 raft_consensus.cc:2454] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e50c7150c0c4355840dcdb446ebabed in term 1.
I20260619 12:26:10.783627 18394 leader_election.cc:302] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e50c7150c0c4355840dcdb446ebabed, e234e3952e454297bab3e40915c753c8; no voters:
I20260619 12:26:10.784116 18581 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a387ac46b64401284412134e9216e5b" candidate_uuid: "1e50c7150c0c4355840dcdb446ebabed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "baa7bb5518cf42c2b0e963a724c5a348"
I20260619 12:26:10.784237 18581 raft_consensus.cc:3037] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:26:10.785571 18804 raft_consensus.cc:2781] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:26:10.785791 18804 raft_consensus.cc:686] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [term 1 LEADER]: Becoming Leader. State: Replica: 1e50c7150c0c4355840dcdb446ebabed, State: Running, Role: LEADER
I20260619 12:26:10.786188 18804 consensus_queue.cc:237] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e50c7150c0c4355840dcdb446ebabed" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 40323 } } peers { permanent_uuid: "baa7bb5518cf42c2b0e963a724c5a348" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 44865 } } peers { permanent_uuid: "e234e3952e454297bab3e40915c753c8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42083 } }
I20260619 12:26:10.786906 18581 raft_consensus.cc:2454] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1e50c7150c0c4355840dcdb446ebabed in term 1.
I20260619 12:26:10.790311 18293 catalog_manager.cc:5495] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed reported cstate change: term changed from 0 to 1, leader changed from <none> to 1e50c7150c0c4355840dcdb446ebabed (127.6.231.193). New cstate: current_term: 1 leader_uuid: "1e50c7150c0c4355840dcdb446ebabed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e50c7150c0c4355840dcdb446ebabed" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 40323 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "baa7bb5518cf42c2b0e963a724c5a348" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 44865 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e234e3952e454297bab3e40915c753c8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42083 } health_report { overall_health: UNKNOWN } } }
I20260619 12:26:10.822901 18791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.815343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38544 (local address 127.6.231.193:40323)
0619 12:26:10.817883 (+ 2540us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:10.817888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:10.817905 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:26:10.817995 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:10.817998 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:10.818065 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:10.818182 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:26:10.818625 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.819407 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.820623 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:10.820933 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:10.821035 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:10.821496 (+ 461us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:26:10.822082 (+ 586us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:26:10.822234 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:10.822696 (+ 462us) server_negotiation.cc:299] Negotiation successful
0619 12:26:10.822758 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2443}
I20260619 12:26:10.830245 18727 raft_consensus.cc:1261] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8 [term 1 FOLLOWER]: Refusing update from remote peer 1e50c7150c0c4355840dcdb446ebabed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:26:10.831154 18807 consensus_queue.cc:1035] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [LEADER]: Connected to new peer: Peer: permanent_uuid: "e234e3952e454297bab3e40915c753c8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:26:10.838820 18581 raft_consensus.cc:1261] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348 [term 1 FOLLOWER]: Refusing update from remote peer 1e50c7150c0c4355840dcdb446ebabed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:26:10.839627 18807 consensus_queue.cc:1035] T 6a387ac46b64401284412134e9216e5b P 1e50c7150c0c4355840dcdb446ebabed [LEADER]: Connected to new peer: Peer: permanent_uuid: "baa7bb5518cf42c2b0e963a724c5a348" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 44865 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:26:10.844643 18810 mvcc.cc:205] Tried to move back new op lower bound from 7298547592511848448 to 7298547592346423296. Current Snapshot: MvccSnapshot[applied={T|T < 7298547592511848448}]
I20260619 12:26:10.849810 18811 mvcc.cc:205] Tried to move back new op lower bound from 7298547592511848448 to 7298547592346423296. Current Snapshot: MvccSnapshot[applied={T|T < 7298547592511848448}]
I20260619 12:26:10.855046 18813 mvcc.cc:205] Tried to move back new op lower bound from 7298547592511848448 to 7298547592346423296. Current Snapshot: MvccSnapshot[applied={T|T < 7298547592511848448}]
I20260619 12:26:10.860618 18292 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:26:10.860801 18292 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:26:10.863893 18292 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2b38a86d0cf84837a85fe5fa547f3c9d: Sending DeleteTablet for 3 replicas of tablet 6a387ac46b64401284412134e9216e5b
I20260619 12:26:10.865213 18440 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a387ac46b64401284412134e9216e5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:26:10 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:38534
I20260619 12:26:10.865661 18576 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a387ac46b64401284412134e9216e5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:26:10 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:49172
I20260619 12:26:10.865749 18711 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a387ac46b64401284412134e9216e5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:26:10 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:45448
I20260619 12:26:10.866887 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 18374
I20260619 12:26:10.869336 18821 tablet_replica.cc:330] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8: stopping tablet replica
I20260619 12:26:10.869758 18821 raft_consensus.cc:2229] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:26:10.870182 18821 raft_consensus.cc:2258] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:26:10.872773 18820 tablet_replica.cc:330] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348: stopping tablet replica
I20260619 12:26:10.873179 18820 raft_consensus.cc:2229] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:26:10.877007 18820 raft_consensus.cc:2258] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:26:10.878175 18821 ts_tablet_manager.cc:1905] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:26:10.878854 18820 ts_tablet_manager.cc:1905] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:26:10.881064 18821 ts_tablet_manager.cc:1918] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:26:10.881177 18821 log.cc:1196] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/6a387ac46b64401284412134e9216e5b
I20260619 12:26:10.881562 18820 ts_tablet_manager.cc:1918] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260619 12:26:10.881867 18277 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:40323 (error 108)
I20260619 12:26:10.882735 18820 log.cc:1196] T 6a387ac46b64401284412134e9216e5b P baa7bb5518cf42c2b0e963a724c5a348: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/6a387ac46b64401284412134e9216e5b
I20260619 12:26:10.883006 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 18513
I20260619 12:26:10.883265 18792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:10.882872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:40323 (local address 127.0.0.1:38552)
0619 12:26:10.883069 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:10.883147 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:40323: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260619 12:26:10.883690 18277 catalog_manager.cc:4584] TS 1e50c7150c0c4355840dcdb446ebabed (127.6.231.193:40323): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a387ac46b64401284412134e9216e5b: Network error: Client connection negotiation failed: client connection to 127.6.231.193:40323: connect: Connection refused (error 111)
I20260619 12:26:10.884581 18821 ts_tablet_manager.cc:1939] T 6a387ac46b64401284412134e9216e5b P e234e3952e454297bab3e40915c753c8: Deleting consensus metadata
I20260619 12:26:10.887820 18277 catalog_manager.cc:4841] TS e234e3952e454297bab3e40915c753c8 (127.6.231.195:42083): tablet 6a387ac46b64401284412134e9216e5b (table test-table [id=6387c899d0f940fc89798f9033927b73]) successfully deleted
W20260619 12:26:10.890722 18792 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:26:10.890262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:44865 (local address 127.0.0.1:49188)
0619 12:26:10.890391 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:10.890408 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:10.890510 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:10.890614 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.194:44865: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 12:26:10.890915 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 18648
I20260619 12:26:10.898671 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 18261
I20260619 12:26:10.906523 7071 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 12:26:11.251703 7071 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 12:26:11.251824 7071 mini_ranger.cc:66] Stopping Ranger...
I20260619 12:26:11.596755 7071 mini_ranger.cc:68] Stopped Ranger
2026-06-19 12:26:11.597 UTC [17847] LOG: received smart shutdown request
2026-06-19 12:26:11.600 UTC [17847] LOG: background worker "logical replication launcher" (PID 17856) exited with exit code 1
2026-06-19 12:26:11.600 UTC [17851] LOG: shutting down
2026-06-19 12:26:11.764 UTC [17847] LOG: database system is shut down
2026-06-19T12:26:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (47883 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:26:11 dist-test-slave-kr9h krb5kdc[18832](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:26:11 dist-test-slave-kr9h krb5kdc[18832](info): set up 2 sockets
Jun 19 12:26:11 dist-test-slave-kr9h krb5kdc[18832](info): commencing operation
krb5kdc: starting...
W20260619 12:26:12.933372 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.027s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:26:13 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781871973, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:26:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:26:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.6.231.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.6.231.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/rangeradmin_127.6.231.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/rangeradmin_127.6.231.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.6.231.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.6.231.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/rangerlookup_127.6.231.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/rangerlookup_127.6.231.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.231.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.6.231.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/HTTP_127.6.231.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/HTTP_127.6.231.212@KRBTEST.COM.keytab.
I20260619 12:26:13.104295 7071 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskYaZBJE/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/postgres -l logfile start
2026-06-19 12:26:15.574 UTC [18862] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-06-19 12:26:15.574 UTC [18862] LOG: listening on IPv4 address "127.6.231.212", port 37957
2026-06-19 12:26:15.578 UTC [18862] LOG: listening on Unix socket "/tmp/.s.PGSQL.37957"
2026-06-19 12:26:15.613 UTC [18865] LOG: database system was shut down at 2026-06-19 12:26:14 UTC
2026-06-19 12:26:15.618 UTC [18862] LOG: database system is ready to accept connections
I20260619 12:26:16.569335 7071 mini_postgres.cc:96] Postgres bound to 37957
2026-06-19 12:26:16.575 UTC [18873] FATAL: database "slave" does not exist
127.6.231.212:37957 - accepting connections
I20260619 12:26:16.576102 7071 mini_ranger.cc:161] Starting Ranger...
I20260619 12:26:16.590915 7071 mini_ranger.cc:84] Created miniranger Postgres user
I20260619 12:26:17.052217 7071 mini_ranger.cc:87] Created ranger Postgres database
I20260619 12:26:17.052299 7071 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-admin
2026-06-19 12:26:17,329 [I] DB FLAVOR :POSTGRES
2026-06-19 12:26:17,329 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 12:26:17,330 [I] Checking connection..
2026-06-19 12:26:17,330 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-06-19 12:26:17,585 [I] Checking connection passed.
2026-06-19 12:26:17,586 [I] --------- Verifying version history table ---------
2026-06-19 12:26:17,586 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:26:17,819 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 12:26:17,820 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:26:18,077 [I] Table x_db_version_h does not exist in database ranger
2026-06-19 12:26:18,077 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-06-19 12:26:18,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-06-19 12:26:18.336 UTC [18937] WARNING: there is no transaction in progress
2026-06-19 12:26:18,342 [I] create_dbversion_catalog.sql file imported successfully
2026-06-19 12:26:18,343 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-06-19 12:26:18,596 [I] Table x_db_version_h already exists in database 'ranger'
2026-06-19 12:26:18,597 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 12:26:18,597 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-06-19 12:26:18,840 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-06-19 12:26:19,078 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-kr9h.c.gcp-upstream.internal','N') ;"
2026-06-19 12:26:19,308 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 12:26:19,556 [I] Table x_portal_user does not exist in database ranger
2026-06-19 12:26:19,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 12:26:19,806 [I] Table x_policy_ref_group does not exist in database ranger
2026-06-19 12:26:19,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 12:26:20,032 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 12:26:20,296 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-06-19 12:26:20,296 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-06-19 12:26:21.240 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.256 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.273 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.289 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.308 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.328 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.393 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.400 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.415 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.426 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.437 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.451 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.461 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.467 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.473 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:21.479 UTC [19045] WARNING: there is no transaction in progress
2026-06-19 12:26:22,103 [I] ranger_core_db_postgres.sql file imported successfully
2026-06-19 12:26:22,103 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-06-19 12:26:22,338 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-06-19 12:26:22,339 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-06-19 12:26:22,590 [I] Table x_portal_user already exists in database 'ranger'
2026-06-19 12:26:22,591 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-06-19 12:26:22,829 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-06-19 12:26:22,829 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-06-19 12:26:23,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-06-19 12:26:23,354 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-kr9h.c.gcp-upstream.internal';"
2026-06-19 12:26:23,604 [I] CORE_DB_SCHEMA import status has been updated
2026-06-19 12:26:23,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-06-19 12:26:23,858 [I] DB_PATCHES have already been applied
I20260619 12:26:23.864343 7071 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-admin:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskYaZBJE/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260619 12:26:23.864405 7071 mini_ranger.cc:193] Using host: 127.6.231.212
I20260619 12:26:23.867784 7071 mini_ranger.cc:227] Ranger admin URL: http://127.6.231.212:36293
Jun 19, 2026 12:26:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 12:26:24 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 12:26:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-admin/ews/webapp
Jun 19, 2026 12:26:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Jun 19, 2026 12:26:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-admin/ews/webapp] .....
Jun 19, 2026 12:26:24 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-admin/ews/webapp].
Jun 19, 2026 12:26:24 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260619 12:26:24.974594 7071 mini_ranger.cc:160] Time spent starting Ranger: real 8.398s user 0.001s sys 0.008s
Jun 19, 2026 12:26:45 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Jun 19, 2026 12:26:45 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Jun 19, 2026 12:26:45 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Jun 19, 2026 12:26:45 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Jun 19, 2026 12:26:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Jun 19, 2026 12:26:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Jun 19, 2026 12:26:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Jun 19, 2026 12:26:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Jun 19, 2026 12:26:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Jun 19, 2026 12:26:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Jun 19, 2026 12:26:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Jun 19, 2026 12:26:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Jun 19, 2026 12:26:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Jun 19, 2026 12:26:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Jun 19, 2026 12:26:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Jun 19, 2026 12:26:45 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Jun 19, 2026 12:26:45 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Jun 19, 2026 12:26:46 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260619 12:26:47.343886 7071 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.6.231.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.6.231.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/rangerkms_127.6.231.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.6.231.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/rangerkms_127.6.231.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.6.231.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.6.231.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.6.231.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/HTTP_127.6.231.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.6.231.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/HTTP_127.6.231.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Jun 19 12:26:47 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872007, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260619 12:26:47.429322 7071 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260619 12:26:47.439332 7071 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260619 12:26:48.292531 7071 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260619 12:26:48.292619 7071 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-kms
I20260619 12:26:48.292639 7071 mini_ranger_kms.cc:227] Using postgres at 127.6.231.212:37957
2026-06-19 12:26:48,468 [I] DB FLAVOR :POSTGRES
2026-06-19 12:26:48,468 [I] --------- Verifying Ranger DB connection ---------
2026-06-19 12:26:48,468 [I] Checking connection
2026-06-19 12:26:48,468 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-06-19 12:26:48,713 [I] connection success
2026-06-19 12:26:48,713 [I] --------- Verifying Ranger DB tables ---------
2026-06-19 12:26:48,713 [I] Verifying table ranger_masterkey in database rangerkms
2026-06-19 12:26:48,713 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-06-19 12:26:48,975 [I] Table ranger_masterkey does not exist in database rangerkms
2026-06-19 12:26:48,975 [I] --------- Importing Ranger Core DB Schema ---------
2026-06-19 12:26:48,976 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-06-19 12:26:48,976 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.6.231.212:37957/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-06-19 12:26:49,228 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260619 12:26:49.734081 7071 mini_ranger_kms.cc:310] Created kms service
I20260619 12:26:49.935911 7071 mini_ranger_kms.cc:326] Created kudu user
I20260619 12:26:49.985704 7071 mini_ranger_kms.cc:343] Created rangerkms user
Jun 19 12:26:49 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872007, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:50.393306 7071 mini_ranger_kms.cc:384] Added ranger policy
I20260619 12:26:50.393584 7071 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-kms:/tmp/dist-test-taskYaZBJE/build/debug/bin/postgresql.jar:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskYaZBJE/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260619 12:26:50.393630 7071 mini_ranger_kms.cc:240] Using host: 127.6.231.212
I20260619 12:26:50.397302 7071 mini_ranger_kms.cc:276] Ranger KMS PID: 19225
I20260619 12:26:50.397396 7071 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.6.231.212:35807
Jun 19, 2026 12:26:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Jun 19, 2026 12:26:51 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Jun 19, 2026 12:26:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Jun 19, 2026 12:26:51 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-kms/ews/webapp] .....
Jun 19, 2026 12:26:52 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-kms/ews/webapp].
Jun 19, 2026 12:26:52 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260619 12:26:52.725871 7071 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.297s user 0.000s sys 0.158s
I20260619 12:26:52.726006 7071 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260619 12:26:52.726078 7071 mini_ranger_kms.cc:401] 127.6.231.212:35807/kms/v1/keys
Jun 19 12:26:52 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781872012, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.231.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Jun 19 12:26:53 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872012, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.231.212@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
Jun 19 12:26:53 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872012, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.6.231.212@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
Jun 19 12:26:55 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:26:55.820092 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:45421
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:45421
--ranger_config_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.231.212:35807/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:26:55.940840 19276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:26:55.941116 19276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:26:55.941155 19276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:26:55.945272 19276 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:26:55.945353 19276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:26:55.945371 19276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:26:55.945389 19276 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:26:55.945403 19276 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:26:55.951020 19276 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/wal
--ranger_kms_url=127.6.231.212:35807/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:45421
--ranger_config_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:45421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19276
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:26:55.952122 19276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:26:55.953208 19276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:26:55.961978 19284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:55.961979 19281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:55.961979 19282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:55.962427 19276 server_base.cc:1034] running on GCE node
I20260619 12:26:55.962986 19276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:26:55.964244 19276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:26:55.965423 19276 hybrid_clock.cc:648] HybridClock initialized: now 1781872015965401 us; error 43 us; skew 500 ppm
Jun 19 12:26:55 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872015, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:55.968407 19276 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:26:55.969350 19276 webserver.cc:460] Webserver started at http://127.6.231.254:46019/ using document root <none> and password file <none>
I20260619 12:26:55.969950 19276 fs_manager.cc:359] Metadata directory not provided
I20260619 12:26:55.970029 19276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:26:55.970254 19276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:26:55 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872015, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:56.019618 19276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "f06f3708030740fd881702c1e6b28fcb"
format_stamp: "Formatted at 2026-06-19 12:26:56 on dist-test-slave-kr9h"
server_key: "f2f40d0b8cea7593e83399c5faeb61b3"
server_key_iv: "495e04e7cbd2c03aa292ccd7a7321d7b"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:56.020371 19276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "f06f3708030740fd881702c1e6b28fcb"
format_stamp: "Formatted at 2026-06-19 12:26:56 on dist-test-slave-kr9h"
server_key: "f2f40d0b8cea7593e83399c5faeb61b3"
server_key_iv: "495e04e7cbd2c03aa292ccd7a7321d7b"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:56.025540 19276 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260619 12:26:56.043926 19291 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:56.045549 19276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:26:56.045712 19276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "f06f3708030740fd881702c1e6b28fcb"
format_stamp: "Formatted at 2026-06-19 12:26:56 on dist-test-slave-kr9h"
server_key: "f2f40d0b8cea7593e83399c5faeb61b3"
server_key_iv: "495e04e7cbd2c03aa292ccd7a7321d7b"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:56.045861 19276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:26:56.062067 19276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:26:56.065229 19276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:26:56.065508 19276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:26:56.073731 19276 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:45421
I20260619 12:26:56.073734 19343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:45421 every 8 connection(s)
I20260619 12:26:56.074810 19276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:26:56.077661 19344 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:26:56.078433 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 19276
I20260619 12:26:56.078559 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:26:56.078891 7071 external_mini_cluster.cc:1428] Setting key d8de2721a6c05fb9c219b3efd0c14b99
I20260619 12:26:56.084441 19344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb: Bootstrap starting.
I20260619 12:26:56.087045 19344 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb: Neither blocks nor log segments found. Creating new log.
I20260619 12:26:56.087939 19344 log.cc:824] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb: Log is configured to *not* fsync() on all Append() calls
Jun 19 12:26:56 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:26:56.090034 19344 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb: No bootstrap required, opened a new log
I20260619 12:26:56.094694 19344 raft_consensus.cc:348] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06f3708030740fd881702c1e6b28fcb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45421 } }
I20260619 12:26:56.094959 19344 raft_consensus.cc:374] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:26:56.095053 19344 raft_consensus.cc:729] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06f3708030740fd881702c1e6b28fcb, State: Initialized, Role: FOLLOWER
I20260619 12:26:56.095396 19347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:56.080366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59938 (local address 127.6.231.254:45421)
0619 12:26:56.080839 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:56.080857 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:56.080900 (+ 43us) server_negotiation.cc:407] Connection header received
0619 12:26:56.081722 (+ 822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:56.081747 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:56.082172 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:56.082531 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:26:56.083377 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:56.084776 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:56.085991 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:56.086647 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:56.086812 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:56.089885 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:26:56.089892 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:26:56.089909 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:26:56.089938 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:26:56.092100 (+ 2162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:56.092633 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:56.092639 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:56.092645 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:56.092787 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:56.093106 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:56.093110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:56.093112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:56.093663 (+ 551us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:26:56.093932 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:56.094155 (+ 223us) server_negotiation.cc:299] Negotiation successful
0619 12:26:56.094437 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":127,"threads_started":1}
I20260619 12:26:56.095582 19344 consensus_queue.cc:260] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06f3708030740fd881702c1e6b28fcb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45421 } }
I20260619 12:26:56.095752 19344 raft_consensus.cc:388] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:26:56.095824 19344 raft_consensus.cc:482] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:26:56.095916 19344 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:26:56.096969 19344 raft_consensus.cc:504] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06f3708030740fd881702c1e6b28fcb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45421 } }
I20260619 12:26:56.097328 19344 leader_election.cc:302] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f06f3708030740fd881702c1e6b28fcb; no voters:
I20260619 12:26:56.097690 19344 leader_election.cc:288] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:26:56.097798 19349 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:26:56.098080 19349 raft_consensus.cc:686] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 1 LEADER]: Becoming Leader. State: Replica: f06f3708030740fd881702c1e6b28fcb, State: Running, Role: LEADER
I20260619 12:26:56.098466 19349 consensus_queue.cc:237] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06f3708030740fd881702c1e6b28fcb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45421 } }
I20260619 12:26:56.098941 19344 sys_catalog.cc:564] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:26:56.099654 19349 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f06f3708030740fd881702c1e6b28fcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06f3708030740fd881702c1e6b28fcb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45421 } } }
I20260619 12:26:56.099836 19349 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [sys.catalog]: This master's current role is: LEADER
I20260619 12:26:56.100063 19349 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f06f3708030740fd881702c1e6b28fcb. Latest consensus state: current_term: 1 leader_uuid: "f06f3708030740fd881702c1e6b28fcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06f3708030740fd881702c1e6b28fcb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45421 } } }
I20260619 12:26:56.100188 19349 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [sys.catalog]: This master's current role is: LEADER
I20260619 12:26:56.101207 19344 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 12:26:56.101567 19358 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:26:56.108456 19358 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:26:56.116253 19358 catalog_manager.cc:1347] Generated new cluster ID: 8b2f9b3f2a6d451d931aada0d1704bdb
I20260619 12:26:56.116441 19358 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:26:56.133261 19358 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:26:56.134824 19358 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:26:56.144600 19358 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb: Generated new TSK 0
I20260619 12:26:56.145761 19358 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:26:56.878825 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.231.212:35807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45421
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:26:57.027221 19389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:26:57.027590 19389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:26:57.027679 19389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:26:57.033609 19389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:26:57.033707 19389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:26:57.033802 19389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:26:57.042062 19389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.6.231.212:35807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:26:57.043671 19389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:26:57.045049 19389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:26:57.056720 19396 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:57.056605 19399 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:57.057332 19389 server_base.cc:1034] running on GCE node
W20260619 12:26:57.057355 19397 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:57.057880 19389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:26:57.058753 19389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:26:57.059937 19389 hybrid_clock.cc:648] HybridClock initialized: now 1781872017059897 us; error 58 us; skew 500 ppm
Jun 19 12:26:57 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872017, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:57.063735 19389 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:26:57.064934 19389 webserver.cc:460] Webserver started at http://127.6.231.193:41963/ using document root <none> and password file <none>
I20260619 12:26:57.065704 19389 fs_manager.cc:359] Metadata directory not provided
I20260619 12:26:57.065855 19389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:26:57.066159 19389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:26:57 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872017, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:57.098155 19389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "f9024da2df31408ba035fcc783750d19"
format_stamp: "Formatted at 2026-06-19 12:26:57 on dist-test-slave-kr9h"
server_key: "4b4b11a6b65ea57d4fb5365e3f48a470"
server_key_iv: "55ea0d9673405cdff06f456472b4dad4"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:57.099085 19389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "f9024da2df31408ba035fcc783750d19"
format_stamp: "Formatted at 2026-06-19 12:26:57 on dist-test-slave-kr9h"
server_key: "4b4b11a6b65ea57d4fb5365e3f48a470"
server_key_iv: "55ea0d9673405cdff06f456472b4dad4"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:57.104836 19389 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 12:26:57.131455 19408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:57.133757 19389 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.002s sys 0.000s
I20260619 12:26:57.133944 19389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "f9024da2df31408ba035fcc783750d19"
format_stamp: "Formatted at 2026-06-19 12:26:57 on dist-test-slave-kr9h"
server_key: "4b4b11a6b65ea57d4fb5365e3f48a470"
server_key_iv: "55ea0d9673405cdff06f456472b4dad4"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:57.134140 19389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:26:57.182269 19389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:26:57.186554 19389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
Jun 19 12:26:57 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781872017, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:57.186940 19389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:26:57.187876 19389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:26:57.189028 19389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:26:57.189096 19389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:57.189157 19389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:26:57.189219 19389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260619 12:26:57.202828 19389 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:46343
I20260619 12:26:57.204066 19389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:26:57.206041 19521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:46343 every 8 connection(s)
I20260619 12:26:57.208549 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 19389
I20260619 12:26:57.208688 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:26:57.209007 7071 external_mini_cluster.cc:1428] Setting key 61613b8c9c748f57659f1c7415628e5a
Jun 19 12:26:57 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872017, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:26:57.225806 19524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:57.208172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:46353 (local address 127.6.231.254:45421)
0619 12:26:57.208460 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:57.208465 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:57.209672 (+ 1207us) server_negotiation.cc:407] Connection header received
0619 12:26:57.210687 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:57.210692 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:57.210757 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:57.210848 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:26:57.212426 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:57.213031 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:57.214377 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:57.214625 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:57.214745 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:57.220996 (+ 6251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:26:57.221003 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:26:57.221009 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:26:57.221049 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:26:57.223323 (+ 2274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:57.223965 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:57.223971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:57.223975 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:57.224084 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:57.224471 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:57.224476 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:57.224481 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:57.224783 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:26:57.224886 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:57.225485 (+ 599us) server_negotiation.cc:299] Negotiation successful
0619 12:26:57.225618 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":98,"threads_started":1}
I20260619 12:26:57.226797 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:57.208637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45421 (local address 127.6.231.193:46353)
0619 12:26:57.209241 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:57.209306 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:57.210424 (+ 1118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:57.211085 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:57.211097 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:57.211485 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:26:57.212256 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:57.212279 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:57.213188 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:57.213192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:57.214253 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:57.214262 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:57.214793 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:57.214798 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:57.214994 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:57.215697 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:26:57.215715 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:26:57.217344 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:26:57.223500 (+ 6156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:57.223506 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:57.223522 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:57.223835 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:57.224179 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:57.224183 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:57.224185 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:57.224354 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:57.224898 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:26:57.224907 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:26:57.225188 (+ 281us) client_negotiation.cc:769] Sending connection context
0619 12:26:57.225389 (+ 201us) client_negotiation.cc:240] Negotiation successful
0619 12:26:57.225827 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":154,"threads_started":1}
I20260619 12:26:57.228060 19522 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45421
I20260619 12:26:57.228331 19522 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:57.228935 19522 heartbeater.cc:511] Master 127.6.231.254:45421 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
I20260619 12:26:57.230916 19308 ts_manager.cc:194] Registered new tserver with Master: f9024da2df31408ba035fcc783750d19 (127.6.231.193:46343)
I20260619 12:26:57.232362 19308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:46353
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:26:57.256393 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.231.212:35807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45421
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:26:57.412892 19528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:26:57.413309 19528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:26:57.413383 19528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:26:57.420526 19528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:26:57.420720 19528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:26:57.420881 19528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:26:57.429723 19528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.6.231.212:35807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:26:57.431294 19528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:26:57.432806 19528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:26:57.444633 19535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:57.444656 19534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:57.444634 19537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:57.445633 19528 server_base.cc:1034] running on GCE node
I20260619 12:26:57.446156 19528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:26:57.447067 19528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:26:57.448305 19528 hybrid_clock.cc:648] HybridClock initialized: now 1781872017448227 us; error 62 us; skew 500 ppm
Jun 19 12:26:57 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872017, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:57.457296 19528 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:26:57.458280 19528 webserver.cc:460] Webserver started at http://127.6.231.194:39449/ using document root <none> and password file <none>
I20260619 12:26:57.458884 19528 fs_manager.cc:359] Metadata directory not provided
I20260619 12:26:57.458945 19528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:26:57.459174 19528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:26:57 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872017, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:57.486086 19528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "696d8c04a63b47d49660294aefca8660"
format_stamp: "Formatted at 2026-06-19 12:26:57 on dist-test-slave-kr9h"
server_key: "ef5593c04fe5a831e079d706127c3495"
server_key_iv: "79cefdfa15495aa30863b900fcc7bdfb"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:57.487076 19528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "696d8c04a63b47d49660294aefca8660"
format_stamp: "Formatted at 2026-06-19 12:26:57 on dist-test-slave-kr9h"
server_key: "ef5593c04fe5a831e079d706127c3495"
server_key_iv: "79cefdfa15495aa30863b900fcc7bdfb"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:57.493168 19528 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.000s
I20260619 12:26:57.522413 19544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:57.524333 19528 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.000s
I20260619 12:26:57.524485 19528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "696d8c04a63b47d49660294aefca8660"
format_stamp: "Formatted at 2026-06-19 12:26:57 on dist-test-slave-kr9h"
server_key: "ef5593c04fe5a831e079d706127c3495"
server_key_iv: "79cefdfa15495aa30863b900fcc7bdfb"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:57.524636 19528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:26:57.552099 19528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:26:57.556000 19528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:26:57.556264 19528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:26:57.557073 19528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:26:57.558199 19528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:26:57.558320 19528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:57.558426 19528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:26:57.558490 19528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:57.573004 19528 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:33301
I20260619 12:26:57.574294 19528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:26:57.575457 19657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:33301 every 8 connection(s)
I20260619 12:26:57.580927 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 19528
I20260619 12:26:57.581061 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:26:57.581389 7071 external_mini_cluster.cc:1428] Setting key c57fb9ea65cf821bca53fd2c38561ebf
Jun 19 12:26:57 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872017, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:26:57.596547 19524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:57.577598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:32945 (local address 127.6.231.254:45421)
0619 12:26:57.579030 (+ 1432us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:57.579035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:57.579480 (+ 445us) server_negotiation.cc:407] Connection header received
0619 12:26:57.580794 (+ 1314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:57.580798 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:57.580866 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:57.580922 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:26:57.582651 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:57.583497 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:57.585461 (+ 1964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:57.585843 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:57.585923 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:57.592208 (+ 6285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:26:57.592213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:26:57.592216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:26:57.592246 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:26:57.594048 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:57.594712 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:57.594716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:57.594718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:57.594803 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:57.595211 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:57.595214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:57.595217 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:57.595439 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:26:57.595518 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:57.596229 (+ 711us) server_negotiation.cc:299] Negotiation successful
0619 12:26:57.596370 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1321}
I20260619 12:26:57.598546 19660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:57.577942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45421 (local address 127.6.231.194:32945)
0619 12:26:57.579310 (+ 1368us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:57.579372 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:57.580525 (+ 1153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:57.581356 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:57.581365 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:57.581765 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:26:57.582477 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:57.582497 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:57.583675 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:57.583681 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:57.585299 (+ 1618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:57.585310 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:57.586132 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:57.586136 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:57.586339 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:57.587050 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:26:57.587073 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:26:57.591971 (+ 4898us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:26:57.594204 (+ 2233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:57.594211 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:57.594232 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:57.594593 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:57.594920 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:57.594923 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:57.594926 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:57.595109 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:57.595585 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:26:57.595592 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:26:57.595946 (+ 354us) client_negotiation.cc:769] Sending connection context
0619 12:26:57.597039 (+ 1093us) client_negotiation.cc:240] Negotiation successful
0619 12:26:57.597407 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1141,"thread_start_us":140,"threads_started":1}
I20260619 12:26:57.600118 19658 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45421
I20260619 12:26:57.600364 19658 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:57.600936 19658 heartbeater.cc:511] Master 127.6.231.254:45421 requested a full tablet report, sending...
I20260619 12:26:57.602172 19308 ts_manager.cc:194] Registered new tserver with Master: 696d8c04a63b47d49660294aefca8660 (127.6.231.194:33301)
I20260619 12:26:57.602964 19308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:32945
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:26:57.637007 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.231.212:35807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45421
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:26:57 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872017, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
W20260619 12:26:57.820278 19663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:26:57.820691 19663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:26:57.820757 19663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:26:57.827358 19663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:26:57.827466 19663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:26:57.827566 19663 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:26:57.836694 19663 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.6.231.212:35807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:45421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:26:57.838258 19663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:26:57.839670 19663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:26:57.852885 19669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:57.853096 19663 server_base.cc:1034] running on GCE node
W20260619 12:26:57.853271 19671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:57.853520 19668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:57.854110 19663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:26:57.855018 19663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:26:57.856230 19663 hybrid_clock.cc:648] HybridClock initialized: now 1781872017856196 us; error 52 us; skew 500 ppm
Jun 19 12:26:57 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872017, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:57.860194 19663 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:26:57.861399 19663 webserver.cc:460] Webserver started at http://127.6.231.195:42277/ using document root <none> and password file <none>
I20260619 12:26:57.862182 19663 fs_manager.cc:359] Metadata directory not provided
I20260619 12:26:57.862270 19663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:26:57.862912 19663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Jun 19 12:26:57 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872017, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:57.890504 19663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "3bc5ee2cff124b03bc261924780ac66d"
format_stamp: "Formatted at 2026-06-19 12:26:57 on dist-test-slave-kr9h"
server_key: "523aa7c59da422747a4f1305705da77e"
server_key_iv: "91f4c8b46b66d6b5891961c8a18918d0"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:57.891391 19663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "3bc5ee2cff124b03bc261924780ac66d"
format_stamp: "Formatted at 2026-06-19 12:26:57 on dist-test-slave-kr9h"
server_key: "523aa7c59da422747a4f1305705da77e"
server_key_iv: "91f4c8b46b66d6b5891961c8a18918d0"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:57.896869 19663 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260619 12:26:57.921931 19678 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:57.923889 19663 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260619 12:26:57.924126 19663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "3bc5ee2cff124b03bc261924780ac66d"
format_stamp: "Formatted at 2026-06-19 12:26:57 on dist-test-slave-kr9h"
server_key: "523aa7c59da422747a4f1305705da77e"
server_key_iv: "91f4c8b46b66d6b5891961c8a18918d0"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:57.924381 19663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:26:57.953383 19663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:26:57.957410 19663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:26:57.957724 19663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:26:57.958572 19663 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
Jun 19 12:26:57 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872017, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:57.960608 19663 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:26:57.960721 19663 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:57.960829 19663 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:26:57.960927 19663 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:57.974869 19663 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:35521
I20260619 12:26:57.978086 19663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:26:57.979866 19792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:35521 every 8 connection(s)
I20260619 12:26:57.984804 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 19663
I20260619 12:26:57.984946 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:26:57.985271 7071 external_mini_cluster.cc:1428] Setting key 78108defb78e085e5065392f5a778d54
Jun 19 12:26:57 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872017, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:26:58.002463 19524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:57.983288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:39779 (local address 127.6.231.254:45421)
0619 12:26:57.983459 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:57.983465 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:57.986752 (+ 3287us) server_negotiation.cc:407] Connection header received
0619 12:26:57.987972 (+ 1220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:57.987975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:57.988026 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:57.988096 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:26:57.991719 (+ 3623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:57.992509 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:57.993944 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:57.994223 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:57.994331 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:57.997920 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:26:57.997924 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:26:57.997927 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:26:57.997957 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:26:57.999728 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:58.000458 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:58.000463 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:58.000467 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:58.000598 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:58.001156 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:58.001160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:58.001162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:58.001405 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:26:58.001541 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:58.002177 (+ 636us) server_negotiation.cc:299] Negotiation successful
0619 12:26:58.002302 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 12:26:58.003484 19795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:57.982246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45421 (local address 127.6.231.195:39779)
0619 12:26:57.986602 (+ 4356us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:57.986657 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:57.987742 (+ 1085us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:57.988713 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:57.988723 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:57.990021 (+ 1298us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:26:57.991568 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:57.991583 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:57.992741 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:57.992744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:57.993821 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:57.993832 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:57.994569 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:57.994578 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:57.994817 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:57.995690 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:26:57.995707 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:26:57.997667 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:26:57.999876 (+ 2209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:57.999892 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:57.999913 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:58.000316 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:58.000817 (+ 501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:58.000821 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:58.000824 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:58.001052 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:58.001546 (+ 494us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:26:58.001557 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:26:58.001926 (+ 369us) client_negotiation.cc:769] Sending connection context
0619 12:26:58.002125 (+ 199us) client_negotiation.cc:240] Negotiation successful
0619 12:26:58.002477 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4060,"thread_start_us":165,"threads_started":1}
I20260619 12:26:58.004709 19793 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45421
I20260619 12:26:58.004994 19793 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:58.005658 19793 heartbeater.cc:511] Master 127.6.231.254:45421 requested a full tablet report, sending...
I20260619 12:26:58.006961 19308 ts_manager.cc:194] Registered new tserver with Master: 3bc5ee2cff124b03bc261924780ac66d (127.6.231.195:35521)
I20260619 12:26:58.007704 19308 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:39779
I20260619 12:26:58.011137 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:26:58.011909 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 19389
I20260619 12:26:58.020483 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 19528
I20260619 12:26:58.027740 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 19663
I20260619 12:26:58.036036 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 19276
I20260619 12:26:58.043169 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:45421
--webserver_interface=127.6.231.254
--webserver_port=46019
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:45421
--ranger_config_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.231.212:35807/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.231.212:35807/kms with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:26:58.190008 19797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:26:58.190296 19797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:26:58.190336 19797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:26:58.194518 19797 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:26:58.194588 19797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:26:58.194607 19797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:26:58.194624 19797 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:26:58.194664 19797 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:26:58.200361 19797 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.231.212:35807/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:45421
--ranger_config_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:45421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=46019
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:26:58.201521 19797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:26:58.202657 19797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:26:58.212678 19805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:58.212693 19803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:58.212693 19802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:58.213198 19797 server_base.cc:1034] running on GCE node
I20260619 12:26:58.213878 19797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:26:58.215236 19797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:26:58.216426 19797 hybrid_clock.cc:648] HybridClock initialized: now 1781872018216416 us; error 48 us; skew 500 ppm
Jun 19 12:26:58 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872018, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:58.219434 19797 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:26:58.220515 19797 webserver.cc:460] Webserver started at http://127.6.231.254:46019/ using document root <none> and password file <none>
I20260619 12:26:58.221134 19797 fs_manager.cc:359] Metadata directory not provided
I20260619 12:26:58.221206 19797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 12:26:58 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872018, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:58.306216 19797 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 12:26:58.309043 19813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:58.310694 19797 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:26:58.310825 19797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "f06f3708030740fd881702c1e6b28fcb"
format_stamp: "Formatted at 2026-06-19 12:26:56 on dist-test-slave-kr9h"
server_key: "f2f40d0b8cea7593e83399c5faeb61b3"
server_key_iv: "495e04e7cbd2c03aa292ccd7a7321d7b"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:58.311520 19797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:26:58.331985 19797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:26:58.335464 19797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:26:58.335691 19797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:26:58.345124 19797 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:45421
I20260619 12:26:58.345150 19865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:45421 every 8 connection(s)
I20260619 12:26:58.346500 19797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:26:58.351075 19866 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:26:58.352802 19866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb: Bootstrap starting.
I20260619 12:26:58.355890 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 19797
I20260619 12:26:58.356381 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.231.212:35807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:46343
--local_ip_for_outbound_sockets=127.6.231.193
--tserver_master_addrs=127.6.231.254:45421
--webserver_port=41963
--webserver_interface=127.6.231.193
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.231.212:35807/kms with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:26:58.374764 19866 log.cc:824] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb: Log is configured to *not* fsync() on all Append() calls
I20260619 12:26:58.383538 19866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:26:58.384016 19866 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb: Bootstrap complete.
I20260619 12:26:58.389547 19866 raft_consensus.cc:348] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06f3708030740fd881702c1e6b28fcb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45421 } }
I20260619 12:26:58.390055 19866 raft_consensus.cc:729] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f06f3708030740fd881702c1e6b28fcb, State: Initialized, Role: FOLLOWER
I20260619 12:26:58.390655 19866 consensus_queue.cc:260] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06f3708030740fd881702c1e6b28fcb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45421 } }
I20260619 12:26:58.390805 19866 raft_consensus.cc:388] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:26:58.390897 19866 raft_consensus.cc:482] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:26:58.390998 19866 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:26:58.393639 19866 raft_consensus.cc:504] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06f3708030740fd881702c1e6b28fcb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45421 } }
I20260619 12:26:58.394011 19866 leader_election.cc:302] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f06f3708030740fd881702c1e6b28fcb; no voters:
I20260619 12:26:58.394368 19866 leader_election.cc:288] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 12:26:58.394507 19870 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 2 FOLLOWER]: Leader election won for term 2
I20260619 12:26:58.394848 19870 raft_consensus.cc:686] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [term 2 LEADER]: Becoming Leader. State: Replica: f06f3708030740fd881702c1e6b28fcb, State: Running, Role: LEADER
I20260619 12:26:58.395256 19870 consensus_queue.cc:237] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06f3708030740fd881702c1e6b28fcb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45421 } }
I20260619 12:26:58.395712 19866 sys_catalog.cc:564] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:26:58.396327 19870 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "f06f3708030740fd881702c1e6b28fcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06f3708030740fd881702c1e6b28fcb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45421 } } }
I20260619 12:26:58.396587 19870 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [sys.catalog]: This master's current role is: LEADER
I20260619 12:26:58.396826 19870 sys_catalog.cc:455] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader f06f3708030740fd881702c1e6b28fcb. Latest consensus state: current_term: 2 leader_uuid: "f06f3708030740fd881702c1e6b28fcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f06f3708030740fd881702c1e6b28fcb" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 45421 } } }
I20260619 12:26:58.396952 19870 sys_catalog.cc:458] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb [sys.catalog]: This master's current role is: LEADER
I20260619 12:26:58.397241 19877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:26:58.400503 19877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:26:58.401021 19866 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260619 12:26:58.401283 19877 catalog_manager.cc:1259] Loaded cluster ID: 8b2f9b3f2a6d451d931aada0d1704bdb
I20260619 12:26:58.401371 19877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:26:58.403793 19877 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:26:58.404472 19877 catalog_manager.cc:5879] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb: Loaded TSK: 0
I20260619 12:26:58.405735 19877 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:26:58.523404 19868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:26:58.523949 19868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:26:58.524107 19868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:26:58.530829 19868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:26:58.531010 19868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:26:58.531160 19868 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:26:58.539734 19868 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.231.212:35807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:46343
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=41963
--tserver_master_addrs=127.6.231.254:45421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:26:58.541386 19868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:26:58.542870 19868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:26:58.556397 19906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:58.557138 19904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:58.557643 19868 server_base.cc:1034] running on GCE node
W20260619 12:26:58.560220 19903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:58.560884 19868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:26:58.561858 19868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:26:58.563162 19868 hybrid_clock.cc:648] HybridClock initialized: now 1781872018563120 us; error 64 us; skew 500 ppm
Jun 19 12:26:58 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872018, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:58.567001 19868 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:26:58.568245 19868 webserver.cc:460] Webserver started at http://127.6.231.193:41963/ using document root <none> and password file <none>
I20260619 12:26:58.569069 19868 fs_manager.cc:359] Metadata directory not provided
I20260619 12:26:58.569245 19868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 12:26:58 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872018, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:58.618717 19868 fs_manager.cc:709] Time spent opening directory manager: real 0.018s user 0.003s sys 0.001s
I20260619 12:26:58.622303 19914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:58.624312 19868 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 12:26:58.624480 19868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "f9024da2df31408ba035fcc783750d19"
format_stamp: "Formatted at 2026-06-19 12:26:57 on dist-test-slave-kr9h"
server_key: "4b4b11a6b65ea57d4fb5365e3f48a470"
server_key_iv: "55ea0d9673405cdff06f456472b4dad4"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:58.625272 19868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:26:58.652001 19868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:26:58.655756 19868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:26:58.656013 19868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:26:58.656734 19868 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:26:58.657827 19868 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:26:58.657913 19868 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:58.657982 19868 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:26:58.658025 19868 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:58.668540 19868 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:46343
I20260619 12:26:58.669965 19868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:26:58.675689 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 19868
I20260619 12:26:58.676190 20027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:46343 every 8 connection(s)
I20260619 12:26:58.676406 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.231.212:35807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:33301
--local_ip_for_outbound_sockets=127.6.231.194
--tserver_master_addrs=127.6.231.254:45421
--webserver_port=39449
--webserver_interface=127.6.231.194
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.231.212:35807/kms with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:26:58 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872018, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:26:58.710225 20031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:58.679567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45421 (local address 127.6.231.193:54393)
0619 12:26:58.684509 (+ 4942us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:58.684578 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:58.685893 (+ 1315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:58.694201 (+ 8308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:58.694211 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:58.694820 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:26:58.695726 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:58.695748 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:58.697965 (+ 2217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:58.697969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:58.699399 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:58.699410 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:58.700718 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:58.700722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:58.700913 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:58.701703 (+ 790us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:26:58.701725 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:26:58.703402 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:26:58.706276 (+ 2874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:58.706283 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:58.706298 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:58.706659 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:58.707099 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:58.707102 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:58.707105 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:58.707313 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:58.708215 (+ 902us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:26:58.708222 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:26:58.708541 (+ 319us) client_negotiation.cc:769] Sending connection context
0619 12:26:58.708754 (+ 213us) client_negotiation.cc:240] Negotiation successful
0619 12:26:58.709110 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4363,"thread_start_us":156,"threads_started":1}
I20260619 12:26:58.711984 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:58.681129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:54393 (local address 127.6.231.254:45421)
0619 12:26:58.684438 (+ 3309us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:58.684461 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:58.685197 (+ 736us) server_negotiation.cc:407] Connection header received
0619 12:26:58.692606 (+ 7409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:58.692624 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:58.693177 (+ 553us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:58.693611 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:26:58.695944 (+ 2333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:58.697798 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:58.699548 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:58.700307 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:58.700479 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:58.703655 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:26:58.703662 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:26:58.703681 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:26:58.703715 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:26:58.706057 (+ 2342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:58.706809 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:58.706817 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:58.706822 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:58.706976 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:58.707434 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:58.707437 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:58.707439 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:58.707977 (+ 538us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:26:58.708144 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:58.710782 (+ 2638us) server_negotiation.cc:299] Negotiation successful
0619 12:26:58.711085 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3139,"thread_start_us":159,"threads_started":1}
I20260619 12:26:58.715618 20028 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45421
I20260619 12:26:58.715895 20028 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:58.716511 20028 heartbeater.cc:511] Master 127.6.231.254:45421 requested a full tablet report, sending...
W20260619 12:26:58.719002 20028 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:45421 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 12:26:58.719138 20028 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:58.719323 20028 heartbeater.cc:511] Master 127.6.231.254:45421 requested a full tablet report, sending...
I20260619 12:26:58.720330 20028 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:58.720494 20028 heartbeater.cc:511] Master 127.6.231.254:45421 requested a full tablet report, sending...
W20260619 12:26:58.721336 20028 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:26:58.883792 20030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:26:58.884212 20030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:26:58.884356 20030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:26:58.890915 20030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:26:58.891052 20030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:26:58.891183 20030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:26:58.899957 20030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.231.212:35807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:33301
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=39449
--tserver_master_addrs=127.6.231.254:45421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:26:58.902153 20030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:26:58.903751 20030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:26:58.917421 20037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:58.917548 20040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:58.917788 20038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:58.918021 20030 server_base.cc:1034] running on GCE node
I20260619 12:26:58.918717 20030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:26:58.919540 20030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:26:58.922847 20030 hybrid_clock.cc:648] HybridClock initialized: now 1781872018922803 us; error 54 us; skew 500 ppm
Jun 19 12:26:58 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872018, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:58.926599 20030 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:26:58.927852 20030 webserver.cc:460] Webserver started at http://127.6.231.194:39449/ using document root <none> and password file <none>
I20260619 12:26:58.928655 20030 fs_manager.cc:359] Metadata directory not provided
I20260619 12:26:58.929085 20030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 12:26:58 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872018, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:58.971783 20030 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.000s sys 0.003s
I20260619 12:26:58.975232 20048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:58.976925 20030 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260619 12:26:58.977057 20030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "696d8c04a63b47d49660294aefca8660"
format_stamp: "Formatted at 2026-06-19 12:26:57 on dist-test-slave-kr9h"
server_key: "ef5593c04fe5a831e079d706127c3495"
server_key_iv: "79cefdfa15495aa30863b900fcc7bdfb"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:58.977861 20030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:26:58.998070 20030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:26:59.002013 20030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:26:59.002255 20030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:26:59.003013 20030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:26:59.004146 20030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:26:59.004221 20030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:59.004287 20030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:26:59.004325 20030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:59.017379 20030 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:33301
I20260619 12:26:59.018839 20030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:26:59.025704 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 20030
I20260619 12:26:59.026269 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.6.231.212:35807/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:35521
--local_ip_for_outbound_sockets=127.6.231.195
--tserver_master_addrs=127.6.231.254:45421
--webserver_port=42277
--webserver_interface=127.6.231.195
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36299
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.6.231.212:35807/kms with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:26:59.026913 20162 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:33301 every 8 connection(s)
Jun 19 12:26:59 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872018, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:26:59.051892 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:59.032727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:55803 (local address 127.6.231.254:45421)
0619 12:26:59.032876 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:59.032882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:59.037986 (+ 5104us) server_negotiation.cc:407] Connection header received
0619 12:26:59.039335 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:59.039339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:59.039406 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:59.039468 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:26:59.041223 (+ 1755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:59.042044 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:59.043676 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:59.044014 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:59.044089 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:59.047333 (+ 3244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:26:59.047337 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:26:59.047340 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:26:59.047369 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:26:59.049277 (+ 1908us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:59.049923 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:59.049927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:59.049929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:59.050016 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:59.050406 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:59.050409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:59.050411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:59.050631 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:26:59.050710 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:59.051433 (+ 723us) server_negotiation.cc:299] Negotiation successful
0619 12:26:59.051574 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:26:59.053908 20166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:59.031487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45421 (local address 127.6.231.194:55803)
0619 12:26:59.037773 (+ 6286us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:59.037842 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:59.039045 (+ 1203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:59.039928 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:59.039940 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:59.040334 (+ 394us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:26:59.041031 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:59.041049 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:59.042217 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:59.042220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:59.043531 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:59.043542 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:59.044309 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:59.044312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:59.044507 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:59.045283 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:26:59.045304 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:26:59.047130 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:26:59.049408 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:59.049414 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:59.049463 (+ 49us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:59.049804 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:59.050132 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:59.050135 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:59.050137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:59.050306 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:59.050774 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:26:59.050781 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:26:59.051096 (+ 315us) client_negotiation.cc:769] Sending connection context
0619 12:26:59.052406 (+ 1310us) client_negotiation.cc:240] Negotiation successful
0619 12:26:59.052780 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6018,"thread_start_us":179,"threads_started":1}
I20260619 12:26:59.055686 20163 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45421
I20260619 12:26:59.055948 20163 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:59.056540 20163 heartbeater.cc:511] Master 127.6.231.254:45421 requested a full tablet report, sending...
W20260619 12:26:59.058389 20163 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:45421 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260619 12:26:59.058517 20163 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:59.058691 20163 heartbeater.cc:511] Master 127.6.231.254:45421 requested a full tablet report, sending...
I20260619 12:26:59.059510 20163 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:59.059667 20163 heartbeater.cc:511] Master 127.6.231.254:45421 requested a full tablet report, sending...
W20260619 12:26:59.060391 20163 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:26:59.203722 20165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:26:59.204121 20165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:26:59.204195 20165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:26:59.210688 20165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:26:59.210788 20165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:26:59.210883 20165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:26:59.219332 20165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36299
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.6.231.212:35807/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:35521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=42277
--tserver_master_addrs=127.6.231.254:45421
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:26:59.220836 20165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:26:59.222280 20165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:26:59.237461 20174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:59.237447 20172 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:26:59.237964 20171 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:26:59.238250 20165 server_base.cc:1034] running on GCE node
I20260619 12:26:59.238831 20165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:26:59.239570 20165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:26:59.240772 20165 hybrid_clock.cc:648] HybridClock initialized: now 1781872019240726 us; error 62 us; skew 500 ppm
Jun 19 12:26:59 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872019, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:59.243952 20165 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:26:59.245133 20165 webserver.cc:460] Webserver started at http://127.6.231.195:42277/ using document root <none> and password file <none>
I20260619 12:26:59.245903 20165 fs_manager.cc:359] Metadata directory not provided
I20260619 12:26:59.245993 20165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Jun 19 12:26:59 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872019, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:26:59.267486 20165 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:26:59.270536 20182 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:59.272166 20165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:26:59.272297 20165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "3bc5ee2cff124b03bc261924780ac66d"
format_stamp: "Formatted at 2026-06-19 12:26:57 on dist-test-slave-kr9h"
server_key: "523aa7c59da422747a4f1305705da77e"
server_key_iv: "91f4c8b46b66d6b5891961c8a18918d0"
server_key_version: "kuduclusterkey@0"
I20260619 12:26:59.273005 20165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:26:59.304353 20165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:26:59.308086 20165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:26:59.308331 20165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:26:59.309084 20165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:26:59.310164 20165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:26:59.310242 20165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:59.310318 20165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:26:59.310374 20165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:26:59.322337 20165 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:35521
I20260619 12:26:59.322356 20302 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:35521 every 8 connection(s)
I20260619 12:26:59.323516 20165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:26:59.327991 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 20165
Jun 19 12:26:59 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872019, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:26:59.340178 20307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:59.328757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59946 (local address 127.6.231.254:45421)
0619 12:26:59.329463 (+ 706us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:59.329472 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:59.329486 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:26:59.329529 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:59.329532 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:59.329573 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:59.330637 (+ 1064us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:26:59.330833 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:59.331656 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:59.333130 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:59.333476 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:59.333583 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:59.335111 (+ 1528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:26:59.335114 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:26:59.335117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:26:59.335142 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:26:59.336944 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:59.338632 (+ 1688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:59.338639 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:59.338641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:59.338725 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:59.339252 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:59.339254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:59.339256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:59.339474 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:26:59.339691 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:59.339863 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 12:26:59.339998 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":602,"thread_start_us":117,"threads_started":1}
I20260619 12:26:59.340875 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:59.325579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:44325 (local address 127.6.231.254:45421)
0619 12:26:59.325718 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 12:26:59.325723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:26:59.326847 (+ 1124us) server_negotiation.cc:407] Connection header received
0619 12:26:59.327948 (+ 1101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:26:59.327952 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:26:59.328017 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:26:59.328078 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:26:59.330119 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:59.330794 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:59.332536 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:26:59.332903 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:26:59.333024 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:59.336527 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:26:59.336531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:26:59.336534 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:26:59.336584 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:26:59.338361 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:59.339041 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:59.339046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:59.339052 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:59.339122 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:26:59.339544 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:26:59.339548 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:26:59.339549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:26:59.339767 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:26:59.339869 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 12:26:59.340570 (+ 701us) server_negotiation.cc:299] Negotiation successful
0619 12:26:59.340729 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:26:59.342578 20305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:26:59.326217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:45421 (local address 127.6.231.195:44325)
0619 12:26:59.326684 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0619 12:26:59.326739 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 12:26:59.327657 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:26:59.328704 (+ 1047us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:26:59.328715 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:26:59.329092 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:26:59.329944 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:59.329966 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:59.330936 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:59.330939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:59.332395 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:26:59.332406 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:26:59.333085 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:26:59.333090 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:26:59.333303 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:26:59.333998 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:26:59.334040 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:26:59.336327 (+ 2287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:26:59.338499 (+ 2172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:59.338504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:59.338518 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:59.338933 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:59.339235 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:26:59.339239 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:26:59.339243 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:26:59.339426 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:26:59.339937 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:26:59.339943 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:26:59.340283 (+ 340us) client_negotiation.cc:769] Sending connection context
0619 12:26:59.341036 (+ 753us) client_negotiation.cc:240] Negotiation successful
0619 12:26:59.341417 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":133,"threads_started":1}
I20260619 12:26:59.343825 20303 heartbeater.cc:347] Connected to a master server at 127.6.231.254:45421
I20260619 12:26:59.344091 20303 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:59.344722 20303 heartbeater.cc:511] Master 127.6.231.254:45421 requested a full tablet report, sending...
I20260619 12:26:59.346246 19830 ts_manager.cc:194] Registered new tserver with Master: 3bc5ee2cff124b03bc261924780ac66d (127.6.231.195:35521)
I20260619 12:26:59.347784 19830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:44325
Jun 19 12:26:59 dist-test-slave-kr9h krb5kdc[18832](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1781872019, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:26:59.721580 20028 heartbeater.cc:464] Registering TS with master...
I20260619 12:26:59.721881 20028 heartbeater.cc:511] Master 127.6.231.254:45421 requested a full tablet report, sending...
I20260619 12:26:59.722648 19830 ts_manager.cc:194] Registered new tserver with Master: f9024da2df31408ba035fcc783750d19 (127.6.231.193:46343)
I20260619 12:26:59.723446 19830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:54393
Jun 19 12:27:00 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872019, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:27:00.060600 20163 heartbeater.cc:464] Registering TS with master...
I20260619 12:27:00.060910 20163 heartbeater.cc:511] Master 127.6.231.254:45421 requested a full tablet report, sending...
I20260619 12:27:00.061677 19830 ts_manager.cc:194] Registered new tserver with Master: 696d8c04a63b47d49660294aefca8660 (127.6.231.194:33301)
I20260619 12:27:00.062464 19830 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:55803
I20260619 12:27:00.071705 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:27:00.086647 20317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:00.075213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51432 (local address 127.6.231.254:45421)
0619 12:27:00.077672 (+ 2459us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:00.077677 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:00.077693 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:27:00.077740 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:00.077743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:00.077791 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:00.077885 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:00.078960 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.079911 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.081083 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.081319 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.081439 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:00.083124 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:00.083127 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:00.083130 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:00.083157 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:00.084876 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:00.085314 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:00.085318 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:00.085321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:00.085413 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:00.085724 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:00.085727 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:00.085729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:00.085933 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:00.086074 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:00.086307 (+ 233us) server_negotiation.cc:299] Negotiation successful
0619 12:27:00.086439 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2373,"thread_start_us":94,"threads_started":1}
I20260619 12:27:00.090689 19830 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51432:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:27:00.094535 19830 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:27:00.114790 20323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:00.104778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:46343 (local address 127.0.0.1:43316)
0619 12:27:00.109814 (+ 5036us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:00.109870 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:00.110038 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:00.110548 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:00.110554 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:00.110578 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:00.110725 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:00.110731 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.111990 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.111994 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:00.112949 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:00.112958 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.114221 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.114225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:00.114348 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:00.114398 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 12:27:00.114486 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 12:27:00.114625 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4844,"thread_start_us":117,"threads_started":1}
I20260619 12:27:00.117640 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:00.105640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43316 (local address 127.6.231.193:46343)
0619 12:27:00.109743 (+ 4103us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:00.109750 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:00.109971 (+ 221us) server_negotiation.cc:407] Connection header received
0619 12:27:00.110212 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:00.110218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:00.110398 (+ 180us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:00.110561 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:00.110874 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.111864 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.113079 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.113831 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.117095 (+ 3264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:00.117296 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:00.117351 (+ 55us) server_negotiation.cc:299] Negotiation successful
0619 12:27:00.117477 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3984,"thread_start_us":72,"threads_started":1}
I20260619 12:27:00.121224 20327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:00.106188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:35521 (local address 127.0.0.1:58800)
0619 12:27:00.113877 (+ 7689us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:00.113893 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:00.114837 (+ 944us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:00.115639 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:00.115645 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:00.115701 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:00.115819 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:00.115829 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.117305 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.117312 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:00.119460 (+ 2148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:00.119471 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.120792 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.120797 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:00.120886 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:00.120912 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 12:27:00.120981 (+ 69us) client_negotiation.cc:240] Negotiation successful
0619 12:27:00.121070 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7617,"mutex_wait_us":27,"thread_start_us":102,"threads_started":1}
I20260619 12:27:00.121824 20328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:00.106572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58800 (local address 127.6.231.195:35521)
0619 12:27:00.106833 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:00.106840 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:00.114751 (+ 7911us) server_negotiation.cc:407] Connection header received
0619 12:27:00.115031 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:00.115038 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:00.115250 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:00.115377 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:00.115963 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.117179 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.119638 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.120422 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.120507 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:00.120687 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:00.121584 (+ 897us) server_negotiation.cc:299] Negotiation successful
0619 12:27:00.121693 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":97,"threads_started":1}
I20260619 12:27:00.121522 19958 tablet_service.cc:1467] Processing CreateTablet for tablet 0a52baa776b74e979b379ed52400b9c5 (DEFAULT_TABLE table=test-table [id=2e08e2486d974e37a369f190e2280a5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:00.122807 19958 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a52baa776b74e979b379ed52400b9c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:00.125962 20324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:00.104900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:33301 (local address 127.0.0.1:33110)
0619 12:27:00.113600 (+ 8700us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:00.113616 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:00.113715 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:00.114520 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:00.114523 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:00.114546 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:00.114658 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:00.114665 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.116090 (+ 1425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.116094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:00.117310 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:00.117322 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.118904 (+ 1582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.118907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:00.125620 (+ 6713us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:00.125652 (+ 32us) client_negotiation.cc:769] Sending connection context
0619 12:27:00.125738 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 12:27:00.125820 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8623,"mutex_wait_us":6530,"thread_start_us":72,"threads_started":1}
I20260619 12:27:00.125576 20230 tablet_service.cc:1467] Processing CreateTablet for tablet 0a52baa776b74e979b379ed52400b9c5 (DEFAULT_TABLE table=test-table [id=2e08e2486d974e37a369f190e2280a5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:00.132185 20329 tablet_bootstrap.cc:492] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19: Bootstrap starting.
I20260619 12:27:00.133635 20230 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a52baa776b74e979b379ed52400b9c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:00.134769 20329 tablet_bootstrap.cc:654] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:00.135699 20329 log.cc:824] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:00.137910 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:00.105094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33110 (local address 127.6.231.194:33301)
0619 12:27:00.113684 (+ 8590us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:00.113691 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:00.113717 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:27:00.113887 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:00.113894 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:00.114118 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:00.114291 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:00.114797 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.115951 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.117659 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.118585 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.118668 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:00.118795 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:00.137600 (+ 18805us) server_negotiation.cc:299] Negotiation successful
0619 12:27:00.137755 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8471,"thread_start_us":112,"threads_started":1}
I20260619 12:27:00.141281 20096 tablet_service.cc:1467] Processing CreateTablet for tablet 0a52baa776b74e979b379ed52400b9c5 (DEFAULT_TABLE table=test-table [id=2e08e2486d974e37a369f190e2280a5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:00.142496 20096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0a52baa776b74e979b379ed52400b9c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:00.143673 20331 tablet_bootstrap.cc:492] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d: Bootstrap starting.
I20260619 12:27:00.146093 20331 tablet_bootstrap.cc:654] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:00.147002 20331 log.cc:824] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:00.149078 20329 tablet_bootstrap.cc:492] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19: No bootstrap required, opened a new log
I20260619 12:27:00.149344 20329 ts_tablet_manager.cc:1397] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19: Time spent bootstrapping tablet: real 0.017s user 0.006s sys 0.000s
I20260619 12:27:00.153666 20329 raft_consensus.cc:348] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9024da2df31408ba035fcc783750d19" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46343 } } peers { permanent_uuid: "696d8c04a63b47d49660294aefca8660" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 33301 } } peers { permanent_uuid: "3bc5ee2cff124b03bc261924780ac66d" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35521 } }
I20260619 12:27:00.153959 20329 raft_consensus.cc:374] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:00.154011 20329 raft_consensus.cc:729] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9024da2df31408ba035fcc783750d19, State: Initialized, Role: FOLLOWER
I20260619 12:27:00.154517 20329 consensus_queue.cc:260] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9024da2df31408ba035fcc783750d19" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46343 } } peers { permanent_uuid: "696d8c04a63b47d49660294aefca8660" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 33301 } } peers { permanent_uuid: "3bc5ee2cff124b03bc261924780ac66d" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35521 } }
I20260619 12:27:00.155606 20329 ts_tablet_manager.cc:1428] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260619 12:27:00.156038 20028 heartbeater.cc:503] Master 127.6.231.254:45421 was elected leader, sending a full tablet report...
I20260619 12:27:00.159293 20331 tablet_bootstrap.cc:492] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d: No bootstrap required, opened a new log
I20260619 12:27:00.159500 20331 ts_tablet_manager.cc:1397] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.000s
I20260619 12:27:00.164115 20331 raft_consensus.cc:348] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9024da2df31408ba035fcc783750d19" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46343 } } peers { permanent_uuid: "696d8c04a63b47d49660294aefca8660" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 33301 } } peers { permanent_uuid: "3bc5ee2cff124b03bc261924780ac66d" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35521 } }
I20260619 12:27:00.164403 20331 raft_consensus.cc:374] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:00.164446 20331 raft_consensus.cc:729] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bc5ee2cff124b03bc261924780ac66d, State: Initialized, Role: FOLLOWER
I20260619 12:27:00.164918 20331 consensus_queue.cc:260] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9024da2df31408ba035fcc783750d19" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46343 } } peers { permanent_uuid: "696d8c04a63b47d49660294aefca8660" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 33301 } } peers { permanent_uuid: "3bc5ee2cff124b03bc261924780ac66d" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35521 } }
I20260619 12:27:00.165658 20303 heartbeater.cc:503] Master 127.6.231.254:45421 was elected leader, sending a full tablet report...
I20260619 12:27:00.166254 20331 ts_tablet_manager.cc:1428] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260619 12:27:00.167055 20333 tablet_bootstrap.cc:492] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660: Bootstrap starting.
I20260619 12:27:00.169627 20333 tablet_bootstrap.cc:654] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:00.170522 20333 log.cc:824] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:00.173473 20333 tablet_bootstrap.cc:492] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660: No bootstrap required, opened a new log
I20260619 12:27:00.173688 20333 ts_tablet_manager.cc:1397] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260619 12:27:00.178112 20333 raft_consensus.cc:348] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9024da2df31408ba035fcc783750d19" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46343 } } peers { permanent_uuid: "696d8c04a63b47d49660294aefca8660" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 33301 } } peers { permanent_uuid: "3bc5ee2cff124b03bc261924780ac66d" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35521 } }
I20260619 12:27:00.178476 20333 raft_consensus.cc:374] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:00.178588 20333 raft_consensus.cc:729] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 696d8c04a63b47d49660294aefca8660, State: Initialized, Role: FOLLOWER
I20260619 12:27:00.179122 20333 consensus_queue.cc:260] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9024da2df31408ba035fcc783750d19" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46343 } } peers { permanent_uuid: "696d8c04a63b47d49660294aefca8660" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 33301 } } peers { permanent_uuid: "3bc5ee2cff124b03bc261924780ac66d" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35521 } }
W20260619 12:27:00.179952 20029 tablet.cc:2369] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:27:00.180413 20333 ts_tablet_manager.cc:1428] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260619 12:27:00.180828 20163 heartbeater.cc:503] Master 127.6.231.254:45421 was elected leader, sending a full tablet report...
I20260619 12:27:00.248142 20337 raft_consensus.cc:482] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:27:00.248401 20337 raft_consensus.cc:504] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9024da2df31408ba035fcc783750d19" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46343 } } peers { permanent_uuid: "696d8c04a63b47d49660294aefca8660" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 33301 } } peers { permanent_uuid: "3bc5ee2cff124b03bc261924780ac66d" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35521 } }
I20260619 12:27:00.250016 20337 leader_election.cc:288] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f9024da2df31408ba035fcc783750d19 (127.6.231.193:46343), 3bc5ee2cff124b03bc261924780ac66d (127.6.231.195:35521)
I20260619 12:27:00.255817 20328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:00.250805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:54503 (local address 127.6.231.195:35521)
0619 12:27:00.250912 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:00.250916 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:00.251192 (+ 276us) server_negotiation.cc:407] Connection header received
0619 12:27:00.251383 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:00.251387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:00.251435 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:00.251484 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:00.252012 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.252786 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.254566 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.255128 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.255219 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:00.255270 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:00.255608 (+ 338us) server_negotiation.cc:299] Negotiation successful
0619 12:27:00.255689 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:27:00.255961 20326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:00.250439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:58943 (local address 127.6.231.193:46343)
0619 12:27:00.250630 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:00.250635 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:00.251148 (+ 513us) server_negotiation.cc:407] Connection header received
0619 12:27:00.251383 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:00.251388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:00.251440 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:00.251492 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:00.252017 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.252802 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.254599 (+ 1797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.255208 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.255303 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:00.255353 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:00.255750 (+ 397us) server_negotiation.cc:299] Negotiation successful
0619 12:27:00.255830 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260619 12:27:00.256407 20338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:00.250264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:46343 (local address 127.6.231.194:58943)
0619 12:27:00.251037 (+ 773us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:00.251065 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:00.251238 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:00.251638 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:00.251640 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:00.251670 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:00.251866 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:00.251875 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.252958 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.252963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:00.254439 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:00.254450 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.255454 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.255458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:00.255596 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:00.255620 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 12:27:00.256178 (+ 558us) client_negotiation.cc:240] Negotiation successful
0619 12:27:00.256259 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"mutex_wait_us":129,"thread_start_us":120,"threads_started":1}
I20260619 12:27:00.256618 20339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:00.250702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:35521 (local address 127.6.231.194:54503)
0619 12:27:00.251098 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:00.251113 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:00.251263 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:00.251631 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:00.251635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:00.251705 (+ 70us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:00.251883 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:00.251889 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.253093 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.253097 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:00.254426 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:00.254437 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.255356 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.255359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:00.255449 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:00.255499 (+ 50us) client_negotiation.cc:769] Sending connection context
0619 12:27:00.256322 (+ 823us) client_negotiation.cc:240] Negotiation successful
0619 12:27:00.256395 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":74,"spinlock_wait_cycles":40192,"thread_start_us":108,"threads_started":1}
I20260619 12:27:00.257392 20233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a52baa776b74e979b379ed52400b9c5" candidate_uuid: "696d8c04a63b47d49660294aefca8660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc5ee2cff124b03bc261924780ac66d" is_pre_election: true
I20260619 12:27:00.257774 20233 raft_consensus.cc:2454] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 696d8c04a63b47d49660294aefca8660 in term 0.
I20260619 12:27:00.258316 20050 leader_election.cc:302] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc5ee2cff124b03bc261924780ac66d, 696d8c04a63b47d49660294aefca8660; no voters:
I20260619 12:27:00.258311 19982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a52baa776b74e979b379ed52400b9c5" candidate_uuid: "696d8c04a63b47d49660294aefca8660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9024da2df31408ba035fcc783750d19" is_pre_election: true
I20260619 12:27:00.258577 19982 raft_consensus.cc:2454] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 696d8c04a63b47d49660294aefca8660 in term 0.
I20260619 12:27:00.258615 20337 raft_consensus.cc:2781] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:27:00.258687 20337 raft_consensus.cc:482] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:27:00.258723 20337 raft_consensus.cc:3037] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:00.260002 20337 raft_consensus.cc:504] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9024da2df31408ba035fcc783750d19" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46343 } } peers { permanent_uuid: "696d8c04a63b47d49660294aefca8660" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 33301 } } peers { permanent_uuid: "3bc5ee2cff124b03bc261924780ac66d" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35521 } }
I20260619 12:27:00.260373 20337 leader_election.cc:288] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [CANDIDATE]: Term 1 election: Requested vote from peers f9024da2df31408ba035fcc783750d19 (127.6.231.193:46343), 3bc5ee2cff124b03bc261924780ac66d (127.6.231.195:35521)
I20260619 12:27:00.260989 19982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a52baa776b74e979b379ed52400b9c5" candidate_uuid: "696d8c04a63b47d49660294aefca8660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f9024da2df31408ba035fcc783750d19"
I20260619 12:27:00.260993 20233 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0a52baa776b74e979b379ed52400b9c5" candidate_uuid: "696d8c04a63b47d49660294aefca8660" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bc5ee2cff124b03bc261924780ac66d"
I20260619 12:27:00.261121 19982 raft_consensus.cc:3037] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:00.261126 20233 raft_consensus.cc:3037] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:00.262418 19982 raft_consensus.cc:2454] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 696d8c04a63b47d49660294aefca8660 in term 1.
I20260619 12:27:00.262480 20233 raft_consensus.cc:2454] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 696d8c04a63b47d49660294aefca8660 in term 1.
I20260619 12:27:00.262799 20050 leader_election.cc:302] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3bc5ee2cff124b03bc261924780ac66d, 696d8c04a63b47d49660294aefca8660; no voters:
I20260619 12:27:00.263028 20337 raft_consensus.cc:2781] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:27:00.263316 20337 raft_consensus.cc:686] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [term 1 LEADER]: Becoming Leader. State: Replica: 696d8c04a63b47d49660294aefca8660, State: Running, Role: LEADER
I20260619 12:27:00.263702 20337 consensus_queue.cc:237] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9024da2df31408ba035fcc783750d19" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46343 } } peers { permanent_uuid: "696d8c04a63b47d49660294aefca8660" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 33301 } } peers { permanent_uuid: "3bc5ee2cff124b03bc261924780ac66d" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35521 } }
I20260619 12:27:00.267381 19830 catalog_manager.cc:5495] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 reported cstate change: term changed from 0 to 1, leader changed from <none> to 696d8c04a63b47d49660294aefca8660 (127.6.231.194). New cstate: current_term: 1 leader_uuid: "696d8c04a63b47d49660294aefca8660" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f9024da2df31408ba035fcc783750d19" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46343 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "696d8c04a63b47d49660294aefca8660" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 33301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3bc5ee2cff124b03bc261924780ac66d" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35521 } health_report { overall_health: UNKNOWN } } }
W20260619 12:27:00.286743 20164 tablet.cc:2369] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:27:00.311159 20325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:00.305308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33116 (local address 127.6.231.194:33301)
0619 12:27:00.305490 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:00.305495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:00.305515 (+ 20us) server_negotiation.cc:407] Connection header received
0619 12:27:00.305710 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:00.305714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:00.305792 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:00.305915 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:27:00.306375 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.307210 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.308621 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:00.308998 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:00.309109 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:00.309627 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:27:00.310251 (+ 624us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:27:00.310391 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:00.310903 (+ 512us) server_negotiation.cc:299] Negotiation successful
0619 12:27:00.310974 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 12:27:00.319701 20233 raft_consensus.cc:1261] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d [term 1 FOLLOWER]: Refusing update from remote peer 696d8c04a63b47d49660294aefca8660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
Jun 19 12:27:00 dist-test-slave-kr9h krb5kdc[18832](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872019, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for HTTP/127.6.231.212@KRBTEST.COM
I20260619 12:27:00.320541 20337 consensus_queue.cc:1035] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bc5ee2cff124b03bc261924780ac66d" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:27:00.320920 19982 raft_consensus.cc:1261] T 0a52baa776b74e979b379ed52400b9c5 P f9024da2df31408ba035fcc783750d19 [term 1 FOLLOWER]: Refusing update from remote peer 696d8c04a63b47d49660294aefca8660: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:27:00.321496 20337 consensus_queue.cc:1035] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f9024da2df31408ba035fcc783750d19" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 46343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260619 12:27:00.325733 20304 tablet.cc:2369] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:27:00.340055 20344 mvcc.cc:205] Tried to move back new op lower bound from 7298547795218481152 to 7298547795002916864. Current Snapshot: MvccSnapshot[applied={T|T < 7298547795218481152}]
I20260619 12:27:00.344790 20345 mvcc.cc:205] Tried to move back new op lower bound from 7298547795218481152 to 7298547795002916864. Current Snapshot: MvccSnapshot[applied={T|T < 7298547795218481152}]
I20260619 12:27:00.350410 20347 mvcc.cc:205] Tried to move back new op lower bound from 7298547795218481152 to 7298547795002916864. Current Snapshot: MvccSnapshot[applied={T|T < 7298547795218481152}]
I20260619 12:27:00.353716 19830 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:27:00.353909 19830 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51432:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:27:00.356860 19830 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f06f3708030740fd881702c1e6b28fcb: Sending DeleteTablet for 3 replicas of tablet 0a52baa776b74e979b379ed52400b9c5
I20260619 12:27:00.357824 19958 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a52baa776b74e979b379ed52400b9c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:00 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:43316
I20260619 12:27:00.357878 20230 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a52baa776b74e979b379ed52400b9c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:00 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:58800
I20260619 12:27:00.358652 20096 tablet_service.cc:1514] Processing DeleteTablet for tablet 0a52baa776b74e979b379ed52400b9c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:00 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:33110
I20260619 12:27:00.359179 20356 tablet_replica.cc:330] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660: stopping tablet replica
I20260619 12:27:00.359591 20356 raft_consensus.cc:2229] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:27:00.359935 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 19868
I20260619 12:27:00.360057 20356 raft_consensus.cc:2258] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:27:00.361841 20356 ts_tablet_manager.cc:1905] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:27:00.364912 20356 ts_tablet_manager.cc:1918] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:27:00.365072 20356 log.cc:1196] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/0a52baa776b74e979b379ed52400b9c5
I20260619 12:27:00.365520 20356 ts_tablet_manager.cc:1939] T 0a52baa776b74e979b379ed52400b9c5 P 696d8c04a63b47d49660294aefca8660: Deleting consensus metadata
I20260619 12:27:00.367007 19816 catalog_manager.cc:4841] TS 696d8c04a63b47d49660294aefca8660 (127.6.231.194:33301): tablet 0a52baa776b74e979b379ed52400b9c5 (table test-table [id=2e08e2486d974e37a369f190e2280a5d]) successfully deleted
I20260619 12:27:00.370417 20354 tablet_replica.cc:330] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d: stopping tablet replica
I20260619 12:27:00.376358 20354 raft_consensus.cc:2229] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:27:00.376966 20354 raft_consensus.cc:2258] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:27:00.379199 20354 ts_tablet_manager.cc:1905] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:27:00.382488 20354 ts_tablet_manager.cc:1918] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260619 12:27:00.383148 19817 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:46343 (error 108)
I20260619 12:27:00.384155 20354 log.cc:1196] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/0a52baa776b74e979b379ed52400b9c5
I20260619 12:27:00.384459 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 20030
I20260619 12:27:00.384688 20354 ts_tablet_manager.cc:1939] T 0a52baa776b74e979b379ed52400b9c5 P 3bc5ee2cff124b03bc261924780ac66d: Deleting consensus metadata
I20260619 12:27:00.386474 19814 catalog_manager.cc:4841] TS 3bc5ee2cff124b03bc261924780ac66d (127.6.231.195:35521): tablet 0a52baa776b74e979b379ed52400b9c5 (table test-table [id=2e08e2486d974e37a369f190e2280a5d]) successfully deleted
I20260619 12:27:00.387274 20324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:00.386875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:46343 (local address 127.0.0.1:43324)
0619 12:27:00.387075 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:00.387177 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:46343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
W20260619 12:27:00.387470 19817 catalog_manager.cc:4584] TS f9024da2df31408ba035fcc783750d19 (127.6.231.193:46343): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0a52baa776b74e979b379ed52400b9c5: Network error: Client connection negotiation failed: client connection to 127.6.231.193:46343: connect: Connection refused (error 111)
I20260619 12:27:00.393577 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 20165
I20260619 12:27:00.401081 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 19797
I20260619 12:27:00.410702 7071 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260619 12:27:00.755901 7071 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260619 12:27:00.756011 7071 mini_ranger.cc:66] Stopping Ranger...
I20260619 12:27:01.101575 7071 mini_ranger.cc:68] Stopped Ranger
2026-06-19 12:27:01.101 UTC [18862] LOG: received smart shutdown request
2026-06-19 12:27:01.105 UTC [18862] LOG: background worker "logical replication launcher" (PID 18871) exited with exit code 1
2026-06-19 12:27:01.106 UTC [18866] LOG: shutting down
2026-06-19 12:27:01.277 UTC [18862] LOG: database system is shut down
2026-06-19T12:27:01Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (49513 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:27:01 dist-test-slave-kr9h krb5kdc[20366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:27:01 dist-test-slave-kr9h krb5kdc[20366](info): set up 2 sockets
Jun 19 12:27:01 dist-test-slave-kr9h krb5kdc[20366](info): commencing operation
krb5kdc: starting...
W20260619 12:27:02.469491 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:27:02 dist-test-slave-kr9h krb5kdc[20366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:27:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:27:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:27:02.593633 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:35437
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:35437,127.6.231.253:45579,127.6.231.252:33343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:27:02.705909 20379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:02.706208 20379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:02.706251 20379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:02.710299 20379 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:27:02.710356 20379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:02.710376 20379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:02.710392 20379 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:27:02.710407 20379 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:27:02.715724 20379 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:35437,127.6.231.253:45579,127.6.231.252:33343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:35437
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20379
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:02.716747 20379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:02.717813 20379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:02.726498 20387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:02.726441 20384 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:02.726651 20379 server_base.cc:1034] running on GCE node
W20260619 12:27:02.726536 20385 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:02.727439 20379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:02.728608 20379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:02.729808 20379 hybrid_clock.cc:648] HybridClock initialized: now 1781872022729787 us; error 33 us; skew 500 ppm
Jun 19 12:27:02 dist-test-slave-kr9h krb5kdc[20366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872022, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:02.736971 20379 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:27:02.738020 20379 webserver.cc:460] Webserver started at http://127.6.231.254:45727/ using document root <none> and password file <none>
I20260619 12:27:02.738615 20379 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:02.738682 20379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:02.738858 20379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:27:02.741178 20379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "dc48e9cc7d9643b3b09a20be8aa29f48"
format_stamp: "Formatted at 2026-06-19 12:27:02 on dist-test-slave-kr9h"
server_key: "30af3ad01ea638d38d35df1f75559dc7"
server_key_iv: "33f4e12cd160af919eced0ca9c45d9b8"
server_key_version: "encryptionkey@0"
I20260619 12:27:02.741834 20379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "dc48e9cc7d9643b3b09a20be8aa29f48"
format_stamp: "Formatted at 2026-06-19 12:27:02 on dist-test-slave-kr9h"
server_key: "30af3ad01ea638d38d35df1f75559dc7"
server_key_iv: "33f4e12cd160af919eced0ca9c45d9b8"
server_key_version: "encryptionkey@0"
I20260619 12:27:02.746166 20379 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:27:02.748809 20394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:02.750363 20379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:27:02.750501 20379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "dc48e9cc7d9643b3b09a20be8aa29f48"
format_stamp: "Formatted at 2026-06-19 12:27:02 on dist-test-slave-kr9h"
server_key: "30af3ad01ea638d38d35df1f75559dc7"
server_key_iv: "33f4e12cd160af919eced0ca9c45d9b8"
server_key_version: "encryptionkey@0"
I20260619 12:27:02.750622 20379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:02.768317 20379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:02.771890 20379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:02.772105 20379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:02.779202 20379 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:35437
I20260619 12:27:02.779214 20446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:35437 every 8 connection(s)
I20260619 12:27:02.780299 20379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:27:02.782991 20447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:02.786751 20447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:27:02.790431 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 20379
I20260619 12:27:02.790586 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:27:02.790936 7071 external_mini_cluster.cc:1428] Setting key 1a8510fa348c12f9a71ff5355f7fb7ed
Jun 19 12:27:02 dist-test-slave-kr9h krb5kdc[20366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872022, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.253@KRBTEST.COM" created.
I20260619 12:27:02.802985 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:02.789045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35437 (local address 127.0.0.1:41594)
0619 12:27:02.789647 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:02.789695 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:02.790499 (+ 804us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:02.791323 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:02.791327 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:02.791643 (+ 316us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:02.791909 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:02.791920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:02.793299 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:02.793305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:02.794669 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:02.794683 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:02.795482 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:02.795489 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:02.795621 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:02.796324 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:02.796340 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:02.798188 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:02.800139 (+ 1951us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:02.800146 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:02.800154 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:02.800373 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:02.800729 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:02.800732 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:02.800733 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:02.800857 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:02.801500 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:02.801505 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:02.801636 (+ 131us) client_negotiation.cc:769] Sending connection context
0619 12:27:02.801780 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 12:27:02.802101 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":434,"mutex_wait_us":68,"thread_start_us":155,"threads_started":1}
I20260619 12:27:02.802991 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:02.789045 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41594 (local address 127.6.231.254:35437)
0619 12:27:02.789601 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:02.789619 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:02.789797 (+ 178us) server_negotiation.cc:407] Connection header received
0619 12:27:02.790866 (+ 1069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:02.790874 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:02.791123 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:02.791322 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:02.792053 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:02.793170 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:02.794809 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:02.795314 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:02.795445 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:02.798332 (+ 2887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:02.798337 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:02.798347 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:02.798368 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:02.800025 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:02.800477 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:02.800481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:02.800483 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:02.800579 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:02.800958 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:02.800961 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:02.800962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:02.801331 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:02.801482 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:02.801860 (+ 378us) server_negotiation.cc:299] Negotiation successful
0619 12:27:02.802021 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"spinlock_wait_cycles":52992,"thread_start_us":159,"threads_started":1}
I20260619 12:27:02.805399 20447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:27:02.806247 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:02.805918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:45579 (local address 127.0.0.1:44630)
0619 12:27:02.806075 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:02.806167 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 12:27:02.806491 20395 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.231.253:45579: connect: Connection refused (error 111)
I20260619 12:27:02.807371 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:02.807099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:45579 (local address 127.0.0.1:44644)
0619 12:27:02.807233 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:02.807294 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260619 12:27:02.807521 20447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.253:45579: Network error: Client connection negotiation failed: client connection to 127.6.231.253:45579: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.253.keytab.
Entry for principal kudu/127.6.231.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.253.keytab.
I20260619 12:27:02.818904 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.253:45579
--webserver_interface=127.6.231.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.253.keytab
--principal=kudu/127.6.231.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:35437,127.6.231.253:45579,127.6.231.252:33343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:27:02.856794 20447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } attempt: 1
I20260619 12:27:02.857731 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:02.857377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:45579 (local address 127.0.0.1:44652)
0619 12:27:02.857560 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:02.857637 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260619 12:27:02.858582 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:02.858350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:45579 (local address 127.0.0.1:44660)
0619 12:27:02.858462 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:02.858517 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 12:27:02.858758 20447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.253:45579: Network error: Client connection negotiation failed: client connection to 127.6.231.253:45579: connect: Connection refused (error 111)
I20260619 12:27:02.926967 20447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } attempt: 2
I20260619 12:27:02.927949 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:02.927556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:45579 (local address 127.0.0.1:44664)
0619 12:27:02.927737 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:02.927816 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260619 12:27:02.928762 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:02.928532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:45579 (local address 127.0.0.1:44676)
0619 12:27:02.928648 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:02.928702 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:45579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260619 12:27:02.928932 20447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.253:45579: Network error: Client connection negotiation failed: client connection to 127.6.231.253:45579: connect: Connection refused (error 111)
W20260619 12:27:02.930253 20454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:02.930541 20454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:02.930641 20454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:02.934660 20454 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:27:02.934736 20454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:02.934796 20454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:02.934857 20454 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:27:02.934894 20454 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:27:02.940784 20454 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:35437,127.6.231.253:45579,127.6.231.252:33343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.253.keytab
--principal=kudu/127.6.231.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.253:45579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.231.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:02.941983 20454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:02.943041 20454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:02.951685 20460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:02.951725 20459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:02.951952 20454 server_base.cc:1034] running on GCE node
W20260619 12:27:02.951670 20462 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:02.952684 20454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:02.953899 20454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:02.955083 20454 hybrid_clock.cc:648] HybridClock initialized: now 1781872022955055 us; error 46 us; skew 500 ppm
Jun 19 12:27:02 dist-test-slave-kr9h krb5kdc[20366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872022, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:02.958002 20454 init.cc:377] Logged in from keytab as kudu/127.6.231.253@KRBTEST.COM (short username kudu)
I20260619 12:27:02.958848 20454 webserver.cc:460] Webserver started at http://127.6.231.253:46557/ using document root <none> and password file <none>
I20260619 12:27:02.959405 20454 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:02.959463 20454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:02.959623 20454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:27:02.961647 20454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/data/instance:
uuid: "cc3f9f46900542308b02539610831a4f"
format_stamp: "Formatted at 2026-06-19 12:27:02 on dist-test-slave-kr9h"
server_key: "78c9892d83a6f7b35548ea8a4922215c"
server_key_iv: "625e95cc6ceba869b6965dffe9c2134a"
server_key_version: "encryptionkey@0"
I20260619 12:27:02.962136 20454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/wal/instance:
uuid: "cc3f9f46900542308b02539610831a4f"
format_stamp: "Formatted at 2026-06-19 12:27:02 on dist-test-slave-kr9h"
server_key: "78c9892d83a6f7b35548ea8a4922215c"
server_key_iv: "625e95cc6ceba869b6965dffe9c2134a"
server_key_version: "encryptionkey@0"
I20260619 12:27:02.965734 20454 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:27:02.968041 20469 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:02.969360 20454 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:27:02.969551 20454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/wal
uuid: "cc3f9f46900542308b02539610831a4f"
format_stamp: "Formatted at 2026-06-19 12:27:02 on dist-test-slave-kr9h"
server_key: "78c9892d83a6f7b35548ea8a4922215c"
server_key_iv: "625e95cc6ceba869b6965dffe9c2134a"
server_key_version: "encryptionkey@0"
I20260619 12:27:02.969662 20454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:02.993938 20454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:02.997099 20454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:02.997277 20454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:03.004540 20454 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.253:45579
I20260619 12:27:03.004562 20521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.253:45579 every 8 connection(s)
I20260619 12:27:03.005546 20454 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/data/info.pb
I20260619 12:27:03.008291 20522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:03.012036 20522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:27:03.015573 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 20454
I20260619 12:27:03.015713 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-1/wal/instance
I20260619 12:27:03.016060 7071 external_mini_cluster.cc:1428] Setting key 52e3a307a98cdd997f62c0a063080b76
I20260619 12:27:03.020119 20447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } attempt: 3
Jun 19 12:27:03 dist-test-slave-kr9h krb5kdc[20366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872022, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.253@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.252@KRBTEST.COM" created.
Jun 19 12:27:03 dist-test-slave-kr9h krb5kdc[20366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872022, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.253@KRBTEST.COM
I20260619 12:27:03.028904 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.013947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41608 (local address 127.6.231.254:35437)
0619 12:27:03.014087 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.014091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.014908 (+ 817us) server_negotiation.cc:407] Connection header received
0619 12:27:03.015968 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.015971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.016020 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.016103 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.017785 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.018701 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.019916 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.020378 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.020501 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.024326 (+ 3825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.024329 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.024332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.024366 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.026514 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.027032 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.027035 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.027036 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.027111 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.027600 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.027604 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.027606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.027811 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.027908 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.028583 (+ 675us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.028728 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:27:03.030172 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.014306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35437 (local address 127.0.0.1:41608)
0619 12:27:03.014751 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.014793 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:03.015730 (+ 937us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:03.016302 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:03.016315 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:03.016856 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:03.017588 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.017611 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.018833 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.018835 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.019790 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.019799 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.020556 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.020561 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.021084 (+ 523us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.022231 (+ 1147us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:03.022251 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:03.024184 (+ 1933us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:03.026652 (+ 2468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.026658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.026670 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.026938 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.027227 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.027229 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.027231 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.027423 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.027938 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:03.027944 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:03.028283 (+ 339us) client_negotiation.cc:769] Sending connection context
0619 12:27:03.028518 (+ 235us) client_negotiation.cc:240] Negotiation successful
0619 12:27:03.028922 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":123,"threads_started":1}
I20260619 12:27:03.031889 20522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:27:03.033656 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.020625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:45579 (local address 127.0.0.1:44684)
0619 12:27:03.020731 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.020748 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:03.020827 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:03.021726 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:03.021729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:03.021935 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:03.022023 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.022029 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.023285 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.023291 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.023952 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.023961 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.024875 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.024880 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.025021 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.025506 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:03.025526 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:03.029354 (+ 3828us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:03.031762 (+ 2408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.031768 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.031773 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.032035 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.032359 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.032363 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.032365 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.032456 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.033026 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:03.033031 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:03.033144 (+ 113us) client_negotiation.cc:769] Sending connection context
0619 12:27:03.033279 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 12:27:03.033472 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260619 12:27:03.033704 20526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.020812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44684 (local address 127.6.231.253:45579)
0619 12:27:03.021219 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.021233 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.021257 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:27:03.021303 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.021311 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.021569 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.021744 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.022138 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.023175 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.024069 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.024547 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.024667 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.029544 (+ 4877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.029553 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.029564 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.029585 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.031641 (+ 2056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.032153 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.032165 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.032167 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.032254 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.032577 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.032580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.032582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.032905 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.033034 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.033352 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.033510 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":205,"threads_started":1}
I20260619 12:27:03.036170 20447 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:27:03 dist-test-slave-kr9h krb5kdc[20366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872022, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.253@KRBTEST.COM for kudu/127.6.231.253@KRBTEST.COM
I20260619 12:27:03.036932 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.036641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36492)
0619 12:27:03.036785 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.036848 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260619 12:27:03.037698 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.037426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36494)
0619 12:27:03.037589 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.037636 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260619 12:27:03.037863 20447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:33343: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
I20260619 12:27:03.041013 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.032440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:45579 (local address 127.0.0.1:44698)
0619 12:27:03.032580 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.032594 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:03.032658 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:03.033168 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:03.033171 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:03.033341 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:03.033418 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.033424 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.034302 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.034308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.034846 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.034853 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.035309 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.035312 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.035396 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.035865 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:03.035879 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:03.037212 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:03.039565 (+ 2353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.039568 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.039569 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.039771 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.040031 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.040033 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.040034 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.040117 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.040465 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:03.040467 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:03.040542 (+ 75us) client_negotiation.cc:769] Sending connection context
0619 12:27:03.040686 (+ 144us) client_negotiation.cc:240] Negotiation successful
0619 12:27:03.040829 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260619 12:27:03.041064 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.032662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44698 (local address 127.6.231.253:45579)
0619 12:27:03.032957 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.032967 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.032981 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:27:03.033013 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.033015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.033069 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.033145 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.033555 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.034196 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.034954 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.035201 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.035303 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.037374 (+ 2071us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.037380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.037385 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.037415 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.039457 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.039853 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.039856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.039857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.039927 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.040215 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.040217 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.040218 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.040378 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.040452 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.040751 (+ 299us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.040893 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":223,"thread_start_us":117,"threads_started":1}
I20260619 12:27:03.041744 20522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:27:03.042520 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.042281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36500)
0619 12:27:03.042390 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.042469 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 12:27:03.042804 20471 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
I20260619 12:27:03.043674 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.043496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36508)
0619 12:27:03.043587 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.043630 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 12:27:03.043840 20522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:33343: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.252.keytab.
Entry for principal kudu/127.6.231.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.252.keytab.
I20260619 12:27:03.048098 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.252:33343
--webserver_interface=127.6.231.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.252.keytab
--principal=kudu/127.6.231.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:45635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:35437,127.6.231.253:45579,127.6.231.252:33343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:27:03.069022 20447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } attempt: 1
I20260619 12:27:03.070001 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.069680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36512)
0619 12:27:03.069843 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.069918 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260619 12:27:03.070845 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.070617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36528)
0619 12:27:03.070725 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.070776 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 12:27:03.071043 20447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:33343: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
I20260619 12:27:03.093138 20522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } attempt: 1
I20260619 12:27:03.094154 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.093843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36538)
0619 12:27:03.094008 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.094075 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260619 12:27:03.094969 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.094769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36542)
0619 12:27:03.094871 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.094917 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 12:27:03.095242 20522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:33343: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
I20260619 12:27:03.146224 20447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } attempt: 2
I20260619 12:27:03.147251 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.146951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36554)
0619 12:27:03.147106 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.147171 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 12:27:03.148031 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.147806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36558)
0619 12:27:03.147921 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.147971 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 12:27:03.148167 20447 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:33343: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
W20260619 12:27:03.161052 20530 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:03.161365 20530 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:03.161495 20530 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 12:27:03.163434 20522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } attempt: 2
I20260619 12:27:03.164338 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.163986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36562)
0619 12:27:03.164137 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.164200 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 12:27:03.165143 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.164908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36564)
0619 12:27:03.165033 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.165086 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260619 12:27:03.165290 20522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:33343: Network error: Client connection negotiation failed: client connection to 127.6.231.252:33343: connect: Connection refused (error 111)
W20260619 12:27:03.166106 20530 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:27:03.166155 20530 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:03.166177 20530 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:03.166230 20530 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:27:03.166257 20530 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:27:03.172154 20530 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:45635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:35437,127.6.231.253:45579,127.6.231.252:33343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.252.keytab
--principal=kudu/127.6.231.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.252:33343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.231.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20530
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:03.173306 20530 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:03.174386 20530 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:03.182653 20538 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:03.182662 20536 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:03.182653 20535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:03.182952 20530 server_base.cc:1034] running on GCE node
I20260619 12:27:03.183651 20530 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:03.184893 20530 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:03.186084 20530 hybrid_clock.cc:648] HybridClock initialized: now 1781872023186056 us; error 51 us; skew 500 ppm
Jun 19 12:27:03 dist-test-slave-kr9h krb5kdc[20366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872023, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:03.188949 20530 init.cc:377] Logged in from keytab as kudu/127.6.231.252@KRBTEST.COM (short username kudu)
I20260619 12:27:03.189985 20530 webserver.cc:460] Webserver started at http://127.6.231.252:34933/ using document root <none> and password file <none>
I20260619 12:27:03.190577 20530 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:03.190667 20530 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:03.190897 20530 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:27:03.192921 20530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/data/instance:
uuid: "2d922b0901d04c6c8b41f75e953eceb7"
format_stamp: "Formatted at 2026-06-19 12:27:03 on dist-test-slave-kr9h"
server_key: "71b8fa0cb8ab17f01a940ed5fbf4ec76"
server_key_iv: "217c34bb30c9c6db145e688e1712adae"
server_key_version: "encryptionkey@0"
I20260619 12:27:03.193555 20530 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/wal/instance:
uuid: "2d922b0901d04c6c8b41f75e953eceb7"
format_stamp: "Formatted at 2026-06-19 12:27:03 on dist-test-slave-kr9h"
server_key: "71b8fa0cb8ab17f01a940ed5fbf4ec76"
server_key_iv: "217c34bb30c9c6db145e688e1712adae"
server_key_version: "encryptionkey@0"
I20260619 12:27:03.197163 20530 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:27:03.199679 20545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:03.200976 20530 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:27:03.201128 20530 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/wal
uuid: "2d922b0901d04c6c8b41f75e953eceb7"
format_stamp: "Formatted at 2026-06-19 12:27:03 on dist-test-slave-kr9h"
server_key: "71b8fa0cb8ab17f01a940ed5fbf4ec76"
server_key_iv: "217c34bb30c9c6db145e688e1712adae"
server_key_version: "encryptionkey@0"
I20260619 12:27:03.201246 20530 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:03.214251 20530 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:03.217321 20530 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:03.217581 20530 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:03.224927 20530 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.252:33343
I20260619 12:27:03.224951 20597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.252:33343 every 8 connection(s)
I20260619 12:27:03.225910 20530 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/data/info.pb
I20260619 12:27:03.228559 20598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:03.232869 20598 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:27:03.234481 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 20530
I20260619 12:27:03.234622 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACert.1781871857195028-7071-0/minicluster-data/master-2/wal/instance
I20260619 12:27:03.234951 7071 external_mini_cluster.cc:1428] Setting key 5b92d02692813dda30be24ffd1dec65c
Jun 19 12:27:03 dist-test-slave-kr9h krb5kdc[20366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872023, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.252@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
Jun 19 12:27:03 dist-test-slave-kr9h krb5kdc[20366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.252@KRBTEST.COM
I20260619 12:27:03.247329 20447 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } attempt: 3
I20260619 12:27:03.247542 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.235470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41616 (local address 127.6.231.254:35437)
0619 12:27:03.236132 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.236139 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.236646 (+ 507us) server_negotiation.cc:407] Connection header received
0619 12:27:03.237931 (+ 1285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.237934 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.237983 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.238058 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.238980 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.239481 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.240532 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.240829 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.240906 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.243670 (+ 2764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.243675 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.243680 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.243715 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.245235 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.245856 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.245858 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.245860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.245943 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.246323 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.246328 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.246332 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.246581 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.246677 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.247254 (+ 577us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.247372 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530}
I20260619 12:27:03.248835 20601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.235859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35437 (local address 127.0.0.1:41616)
0619 12:27:03.236501 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.236554 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:03.237704 (+ 1150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:03.238128 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:03.238133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:03.238549 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:03.238825 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.238845 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.239585 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.239588 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.240423 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.240430 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.240958 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.240962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.241135 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.241827 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:03.241842 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:03.243450 (+ 1608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:03.245379 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.245387 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.245398 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.245764 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.246035 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.246040 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.246044 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.246222 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.246685 (+ 463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:03.246694 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:03.247005 (+ 311us) client_negotiation.cc:769] Sending connection context
0619 12:27:03.247198 (+ 193us) client_negotiation.cc:240] Negotiation successful
0619 12:27:03.247507 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":133,"threads_started":1}
I20260619 12:27:03.248854 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.236189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36572 (local address 127.6.231.252:33343)
0619 12:27:03.236937 (+ 748us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.236949 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.236992 (+ 43us) server_negotiation.cc:407] Connection header received
0619 12:27:03.237704 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.237728 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.237972 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.238166 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.238867 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.239863 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.240727 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.241173 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.241288 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.244068 (+ 2780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.244076 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.244088 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.244114 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.246141 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.246703 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.246709 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.246711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.246808 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.247136 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.247138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.247139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.247438 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.248351 (+ 913us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.248456 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.248617 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":248,"threads_started":1}
I20260619 12:27:03.250057 20598 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:27:03 dist-test-slave-kr9h krb5kdc[20366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872022, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.252@KRBTEST.COM
Jun 19 12:27:03 dist-test-slave-kr9h krb5kdc[20366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872023, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.252@KRBTEST.COM for kudu/127.6.231.253@KRBTEST.COM
I20260619 12:27:03.256446 20522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } attempt: 3
Jun 19 12:27:03 dist-test-slave-kr9h krb5kdc[20366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:27:03.258924 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.247959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36588 (local address 127.6.231.252:33343)
0619 12:27:03.248246 (+ 287us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.248255 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.248305 (+ 50us) server_negotiation.cc:407] Connection header received
0619 12:27:03.248430 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.248433 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.248473 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.248566 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.249001 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.251096 (+ 2095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.252155 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.252474 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.252580 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.255164 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.255168 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.255171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.255196 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.256901 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.257460 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.257466 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.257471 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.257566 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.257894 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.257897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.257898 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.258116 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.258210 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.258590 (+ 380us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.258722 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":110,"threads_started":1}
I20260619 12:27:03.258917 20449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.247867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36588)
0619 12:27:03.248247 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.248262 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:03.248342 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:03.248602 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:03.248605 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:03.248765 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:03.248870 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.248879 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.251190 (+ 2311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.251193 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.252029 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.252039 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.252607 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.252611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.252694 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.253321 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:03.253336 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:03.255020 (+ 1684us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:03.257040 (+ 2020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.257044 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.257046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.257314 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.257681 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.257685 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.257686 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.257777 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.258200 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:03.258203 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:03.258300 (+ 97us) client_negotiation.cc:769] Sending connection context
0619 12:27:03.258423 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 12:27:03.258722 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288}
I20260619 12:27:03.261343 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.250670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44706 (local address 127.6.231.253:45579)
0619 12:27:03.250788 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.250793 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.250808 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:27:03.250879 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.250882 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.250939 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.251028 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.251446 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.252184 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.253767 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.254076 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.254190 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.257383 (+ 3193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.257388 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.257393 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.257463 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.259471 (+ 2008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.259979 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.259986 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.259990 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.260074 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.260407 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.260410 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.260412 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.260627 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.260701 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.261049 (+ 348us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.261174 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:27:03.262120 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.251412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41620 (local address 127.6.231.254:35437)
0619 12:27:03.251511 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.251515 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.251528 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:27:03.251559 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.251561 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.251590 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.251682 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.252292 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.252995 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.254175 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.254494 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.254618 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.257873 (+ 3255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.257878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.257882 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.257904 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.259824 (+ 1920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.260324 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.260330 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.260335 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.260416 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.260721 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.260727 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.260732 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.260936 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.261125 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.261843 (+ 718us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.261955 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 12:27:03.262120 20601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.250564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:45579 (local address 127.0.0.1:44706)
0619 12:27:03.250709 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.250723 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:03.250790 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:03.251039 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:03.251042 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:03.251204 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:03.251299 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.251305 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.252299 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.252302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.253648 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.253657 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.254311 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.254316 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.254427 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.254871 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:03.254886 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:03.257231 (+ 2345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:03.259592 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.259595 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.259597 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.259850 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.260196 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.260199 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.260201 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.260289 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.260778 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:03.260780 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:03.260864 (+ 84us) client_negotiation.cc:769] Sending connection context
0619 12:27:03.261824 (+ 960us) client_negotiation.cc:240] Negotiation successful
0619 12:27:03.261951 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
Jun 19 12:27:03 dist-test-slave-kr9h krb5kdc[20366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872022, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.253@KRBTEST.COM for kudu/127.6.231.252@KRBTEST.COM
I20260619 12:27:03.263125 20598 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:27:03.263283 20447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48: Bootstrap starting.
I20260619 12:27:03.266822 20524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.256870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36602)
0619 12:27:03.257099 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.257115 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:03.257199 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:03.257457 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:03.257461 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:03.257789 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:03.257897 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.257907 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.259785 (+ 1878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.259789 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.260541 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.260550 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.261176 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.261179 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.261255 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.261732 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:03.261754 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:03.263191 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:03.265221 (+ 2030us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.265224 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.265226 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.265482 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.265797 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.265800 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.265802 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.265896 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.266338 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:03.266341 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:03.266435 (+ 94us) client_negotiation.cc:769] Sending connection context
0619 12:27:03.266523 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 12:27:03.266649 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260619 12:27:03.267351 20602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.256956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36602 (local address 127.6.231.252:33343)
0619 12:27:03.257061 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.257065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.257176 (+ 111us) server_negotiation.cc:407] Connection header received
0619 12:27:03.257274 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.257280 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.257328 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.257585 (+ 257us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.258383 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.259664 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.260667 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.261043 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.261469 (+ 426us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.263338 (+ 1869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.263342 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.263345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.263372 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.265102 (+ 1730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.265611 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.265615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.265616 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.265698 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.266013 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.266016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.266021 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.266226 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.267046 (+ 820us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.267119 (+ 73us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.267219 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 12:27:03.267596 20447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48: Neither blocks nor log segments found. Creating new log.
Jun 19 12:27:03 dist-test-slave-kr9h krb5kdc[20366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872023, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.252@KRBTEST.COM for kudu/127.6.231.252@KRBTEST.COM
I20260619 12:27:03.268452 20447 log.cc:824] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:03.270653 20447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48: No bootstrap required, opened a new log
I20260619 12:27:03.270723 20522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f: Bootstrap starting.
I20260619 12:27:03.272079 20601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.263653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:33343 (local address 127.0.0.1:36606)
0619 12:27:03.263779 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:03.263793 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:03.263860 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:03.264196 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:03.264199 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:03.264372 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:03.264585 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.264591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.265287 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.265293 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.265982 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:03.265990 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.266484 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.266487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:03.266633 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.267250 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:03.267263 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:03.268691 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:03.270638 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.270640 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.270643 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.270872 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.271147 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:03.271150 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:03.271152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:03.271222 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:03.271611 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:03.271614 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:03.271680 (+ 66us) client_negotiation.cc:769] Sending connection context
0619 12:27:03.271768 (+ 88us) client_negotiation.cc:240] Negotiation successful
0619 12:27:03.271885 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":174}
I20260619 12:27:03.272886 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.263941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36606 (local address 127.6.231.252:33343)
0619 12:27:03.264025 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.264029 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.264041 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:27:03.264072 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.264074 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.264103 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.264188 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.264698 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.265187 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.266065 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.266389 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.266488 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.268813 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.268817 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.268819 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.268839 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.270526 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.270972 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.270985 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.270987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.271060 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.271304 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.271307 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.271309 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.271497 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.271581 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.272377 (+ 796us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.272482 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":25,"spinlock_wait_cycles":3968}
I20260619 12:27:03.274403 20522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f: Neither blocks nor log segments found. Creating new log.
Jun 19 12:27:03 dist-test-slave-kr9h krb5kdc[20366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872022, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.253@KRBTEST.COM
I20260619 12:27:03.275319 20522 log.cc:824] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:03.275475 20447 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } }
I20260619 12:27:03.275796 20447 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:03.275898 20447 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc48e9cc7d9643b3b09a20be8aa29f48, State: Initialized, Role: FOLLOWER
I20260619 12:27:03.276367 20598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7: Bootstrap starting.
I20260619 12:27:03.276479 20447 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } }
I20260619 12:27:03.277697 20522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f: No bootstrap required, opened a new log
I20260619 12:27:03.277938 20447 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:27:03.278864 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.267075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44718 (local address 127.6.231.253:45579)
0619 12:27:03.267180 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.267185 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.268969 (+ 1784us) server_negotiation.cc:407] Connection header received
0619 12:27:03.269156 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.269159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.269225 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.269294 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.270115 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.270799 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.271823 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.272131 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.272201 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.275198 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.275202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.275205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.275229 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.276862 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.277333 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.277337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.277338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.277421 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.277764 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.277768 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.277769 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.277975 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.278126 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.278549 (+ 423us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.278669 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260619 12:27:03.278908 20598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:03.279734 20598 log.cc:824] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:03.279865 20606 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } } }
I20260619 12:27:03.279984 20606 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:27:03.282615 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:27:03.283589 20598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7: No bootstrap required, opened a new log
W20260619 12:27:03.285256 20617 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:27:03.285351 20617 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:27:03.283102 20379 master_runner.cc:186] Error getting master registration for 127.6.231.253:45579: OK, instance_id { permanent_uuid: "cc3f9f46900542308b02539610831a4f" instance_seqno: 1781872023002400 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:27:03.286440 20379 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:27:03.287526 20522 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } }
I20260619 12:27:03.287549 20598 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } }
I20260619 12:27:03.287760 20598 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:03.287820 20598 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d922b0901d04c6c8b41f75e953eceb7, State: Initialized, Role: FOLLOWER
I20260619 12:27:03.287928 20522 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:03.288022 20522 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc3f9f46900542308b02539610831a4f, State: Initialized, Role: FOLLOWER
I20260619 12:27:03.288357 20598 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } }
I20260619 12:27:03.288458 20522 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } }
I20260619 12:27:03.289731 20522 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:27:03.289731 20598 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:27:03.299142 20530 master_runner.cc:263] No leader master found from master 2d922b0901d04c6c8b41f75e953eceb7
W20260619 12:27:03.299157 20648 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260619 12:27:03.299216 20530 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260619 12:27:03.299216 20648 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:27:03.299489 20627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } } }
I20260619 12:27:03.299688 20627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:27:03.299784 20454 master_runner.cc:263] No leader master found from master cc3f9f46900542308b02539610831a4f
I20260619 12:27:03.299847 20454 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:27:03.300016 20628 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } } }
I20260619 12:27:03.300177 20628 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [sys.catalog]: This master's current role is: FOLLOWER
W20260619 12:27:03.303355 20650 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:27:03.303483 20650 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:27:03.311132 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.286176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36614 (local address 127.6.231.252:33343)
0619 12:27:03.286270 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.286276 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.287172 (+ 896us) server_negotiation.cc:407] Connection header received
0619 12:27:03.288838 (+ 1666us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.288841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.288883 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.288930 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.290739 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.291522 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.297613 (+ 6091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.297950 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.298029 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.302345 (+ 4316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.302349 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.302352 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.302377 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.304288 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.306216 (+ 1928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.306219 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.306221 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.306315 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.308462 (+ 2147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.308466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.308468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.308692 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.308826 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.310832 (+ 2006us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.310963 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260619 12:27:03.311985 20528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.286039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44722 (local address 127.6.231.253:45579)
0619 12:27:03.286153 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.286158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.286996 (+ 838us) server_negotiation.cc:407] Connection header received
0619 12:27:03.288606 (+ 1610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.288611 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.288656 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.288708 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.294729 (+ 6021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.295476 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.298551 (+ 3075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.298886 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.298960 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.302718 (+ 3758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.302723 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.302725 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.302751 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.305026 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.306879 (+ 1853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.306883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.306885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.306960 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.309576 (+ 2616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.309586 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.309588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.309797 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.309903 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.311704 (+ 1801us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.311831 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:27:03.313726 20450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:03.285835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.6.231.254:35437)
0619 12:27:03.285951 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:03.285956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:03.286663 (+ 707us) server_negotiation.cc:407] Connection header received
0619 12:27:03.287259 (+ 596us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:03.287262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:03.287314 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:03.287366 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:03.291664 (+ 4298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.292429 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.296622 (+ 4193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:03.296954 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:03.297046 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:03.305873 (+ 8827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:03.305877 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:03.305880 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:03.305908 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:03.307693 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.308902 (+ 1209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.308906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.308908 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.308989 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:03.310179 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:03.310182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:03.310184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:03.310408 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:03.310510 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:03.313414 (+ 2904us) server_negotiation.cc:299] Negotiation successful
0619 12:27:03.313572 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:27:03.461503 20606 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:27:03.461742 20606 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } }
I20260619 12:27:03.462802 20606 leader_election.cc:288] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cc3f9f46900542308b02539610831a4f (127.6.231.253:45579), 2d922b0901d04c6c8b41f75e953eceb7 (127.6.231.252:33343)
I20260619 12:27:03.463414 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d922b0901d04c6c8b41f75e953eceb7" is_pre_election: true
I20260619 12:27:03.463411 20496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc3f9f46900542308b02539610831a4f" is_pre_election: true
I20260619 12:27:03.463722 20496 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc48e9cc7d9643b3b09a20be8aa29f48 in term 0.
I20260619 12:27:03.463722 20572 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dc48e9cc7d9643b3b09a20be8aa29f48 in term 0.
I20260619 12:27:03.464143 20395 leader_election.cc:302] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cc3f9f46900542308b02539610831a4f, dc48e9cc7d9643b3b09a20be8aa29f48; no voters:
I20260619 12:27:03.464473 20606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:27:03.464550 20606 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:27:03.464598 20606 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:03.466179 20606 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } }
I20260619 12:27:03.467061 20572 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d922b0901d04c6c8b41f75e953eceb7"
I20260619 12:27:03.467180 20572 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:03.467953 20496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cc3f9f46900542308b02539610831a4f"
I20260619 12:27:03.468079 20496 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:03.468581 20572 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc48e9cc7d9643b3b09a20be8aa29f48 in term 1.
W20260619 12:27:03.468894 20551 tablet.cc:2369] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:27:03.468984 20396 leader_election.cc:302] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d922b0901d04c6c8b41f75e953eceb7, dc48e9cc7d9643b3b09a20be8aa29f48; no voters:
I20260619 12:27:03.469511 20496 raft_consensus.cc:2454] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dc48e9cc7d9643b3b09a20be8aa29f48 in term 1.
I20260619 12:27:03.469588 20606 leader_election.cc:288] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [CANDIDATE]: Term 1 election: Requested vote from peers cc3f9f46900542308b02539610831a4f (127.6.231.253:45579), 2d922b0901d04c6c8b41f75e953eceb7 (127.6.231.252:33343)
I20260619 12:27:03.469686 20606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:27:03.469803 20606 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [term 1 LEADER]: Becoming Leader. State: Replica: dc48e9cc7d9643b3b09a20be8aa29f48, State: Running, Role: LEADER
I20260619 12:27:03.470217 20606 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } }
I20260619 12:27:03.472834 20653 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc48e9cc7d9643b3b09a20be8aa29f48. Latest consensus state: current_term: 1 leader_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } } }
I20260619 12:27:03.472952 20653 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:03.473492 20655 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:27:03.474390 20655 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:27:03.479104 20496 raft_consensus.cc:1261] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [term 1 FOLLOWER]: Refusing update from remote peer dc48e9cc7d9643b3b09a20be8aa29f48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:27:03.479171 20572 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [term 1 FOLLOWER]: Refusing update from remote peer dc48e9cc7d9643b3b09a20be8aa29f48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:27:03.479866 20653 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:27:03.480010 20606 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:27:03.487919 20627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc48e9cc7d9643b3b09a20be8aa29f48. Latest consensus state: current_term: 1 leader_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } } }
I20260619 12:27:03.488092 20627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:27:03.491817 20628 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc48e9cc7d9643b3b09a20be8aa29f48. Latest consensus state: current_term: 1 leader_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } } }
I20260619 12:27:03.491986 20628 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:27:03.497684 20653 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } } }
I20260619 12:27:03.497839 20653 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:03.497962 20653 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } } }
I20260619 12:27:03.498040 20653 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48 [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:03.498020 20628 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } } }
I20260619 12:27:03.498144 20656 mvcc.cc:205] Tried to move back new op lower bound from 7298547808162701312 to 7298547808138805248. Current Snapshot: MvccSnapshot[applied={T|T < 7298547808162701312}]
I20260619 12:27:03.498178 20628 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:27:03.498292 20659 mvcc.cc:205] Tried to move back new op lower bound from 7298547808162701312 to 7298547808138805248. Current Snapshot: MvccSnapshot[applied={T|T < 7298547808162701312}]
I20260619 12:27:03.504338 20627 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dc48e9cc7d9643b3b09a20be8aa29f48" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35437 } } peers { permanent_uuid: "cc3f9f46900542308b02539610831a4f" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 45579 } } peers { permanent_uuid: "2d922b0901d04c6c8b41f75e953eceb7" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 33343 } } }
I20260619 12:27:03.504429 20657 mvcc.cc:205] Tried to move back new op lower bound from 7298547808162701312 to 7298547808138805248. Current Snapshot: MvccSnapshot[applied={T|T < 7298547808162701312}]
I20260619 12:27:03.504510 20627 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:27:03.505604 20655 catalog_manager.cc:1347] Generated new cluster ID: 9d0aad3994dd4b9a9169cb44a4c9ca23
I20260619 12:27:03.505720 20655 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:27:03.535629 20655 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:27:03.537196 20655 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:27:03.544600 20655 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc48e9cc7d9643b3b09a20be8aa29f48: Generated new TSK 0
I20260619 12:27:03.545612 20655 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:27:04.300499 20648 catalog_manager.cc:1259] Loaded cluster ID: 9d0aad3994dd4b9a9169cb44a4c9ca23
I20260619 12:27:04.300590 20648 catalog_manager.cc:1552] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f: loading cluster ID for follower catalog manager: success
I20260619 12:27:04.302342 20648 catalog_manager.cc:1574] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f: acquiring CA information for follower catalog manager: success
I20260619 12:27:04.303509 20648 catalog_manager.cc:1602] T 00000000000000000000000000000000 P cc3f9f46900542308b02539610831a4f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:27:04.304559 20650 catalog_manager.cc:1259] Loaded cluster ID: 9d0aad3994dd4b9a9169cb44a4c9ca23
I20260619 12:27:04.304648 20650 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7: loading cluster ID for follower catalog manager: success
I20260619 12:27:04.306296 20650 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7: acquiring CA information for follower catalog manager: success
I20260619 12:27:04.307392 20650 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2d922b0901d04c6c8b41f75e953eceb7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:27:04.605965 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 20379
I20260619 12:27:04.612233 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 20454
I20260619 12:27:04.619614 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 20530
2026-06-19T12:27:04Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3221 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:27:04 dist-test-slave-kr9h krb5kdc[20671](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:27:04 dist-test-slave-kr9h krb5kdc[20671](info): set up 2 sockets
Jun 19 12:27:04 dist-test-slave-kr9h krb5kdc[20671](info): commencing operation
krb5kdc: starting...
W20260619 12:27:05.709690 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:27:05 dist-test-slave-kr9h krb5kdc[20671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:27:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:27:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:27:05.833236 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:41553
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:42739
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:41553
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:27:05.950312 20684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:05.950625 20684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:05.950700 20684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:05.954876 20684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:27:05.954967 20684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:05.955004 20684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:05.955055 20684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:27:05.955091 20684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:27:05.960352 20684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:42739
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:41553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:41553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:05.961524 20684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:05.962536 20684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:05.970809 20690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:05.970880 20689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:05.970813 20692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:05.971006 20684 server_base.cc:1034] running on GCE node
I20260619 12:27:05.971843 20684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:05.972949 20684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:05.974159 20684 hybrid_clock.cc:648] HybridClock initialized: now 1781872025974121 us; error 55 us; skew 500 ppm
Jun 19 12:27:05 dist-test-slave-kr9h krb5kdc[20671](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872025, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:05.976883 20684 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:27:05.977849 20684 webserver.cc:460] Webserver started at http://127.6.231.254:42453/ using document root <none> and password file <none>
I20260619 12:27:05.978394 20684 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:05.978487 20684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:05.978718 20684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:27:05.980751 20684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "13842b92ffd84fb6997e79733775466a"
format_stamp: "Formatted at 2026-06-19 12:27:05 on dist-test-slave-kr9h"
server_key: "7d6dd9ef92a661462a6c80b64e5f5d51"
server_key_iv: "d8d17fc8ed2a8fab598c08a352d87d66"
server_key_version: "encryptionkey@0"
I20260619 12:27:05.981346 20684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "13842b92ffd84fb6997e79733775466a"
format_stamp: "Formatted at 2026-06-19 12:27:05 on dist-test-slave-kr9h"
server_key: "7d6dd9ef92a661462a6c80b64e5f5d51"
server_key_iv: "d8d17fc8ed2a8fab598c08a352d87d66"
server_key_version: "encryptionkey@0"
I20260619 12:27:05.984921 20684 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:27:05.987423 20699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:05.988613 20684 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260619 12:27:05.988752 20684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "13842b92ffd84fb6997e79733775466a"
format_stamp: "Formatted at 2026-06-19 12:27:05 on dist-test-slave-kr9h"
server_key: "7d6dd9ef92a661462a6c80b64e5f5d51"
server_key_iv: "d8d17fc8ed2a8fab598c08a352d87d66"
server_key_version: "encryptionkey@0"
I20260619 12:27:05.988880 20684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:06.009693 20684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:06.012755 20684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:06.012979 20684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:06.020164 20684 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:41553
I20260619 12:27:06.020231 20751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:41553 every 8 connection(s)
I20260619 12:27:06.021167 20684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:27:06.024055 20752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:06.029626 20752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a: Bootstrap starting.
I20260619 12:27:06.030115 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 20684
I20260619 12:27:06.030236 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:27:06.030527 7071 external_mini_cluster.cc:1428] Setting key 5747f3c5b88c4b6c0046aa9c6475777b
I20260619 12:27:06.032388 20752 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:06.033279 20752 log.cc:824] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:06.035677 20752 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a: No bootstrap required, opened a new log
I20260619 12:27:06.039739 20752 raft_consensus.cc:348] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13842b92ffd84fb6997e79733775466a" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41553 } }
I20260619 12:27:06.040026 20752 raft_consensus.cc:374] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:06.040077 20752 raft_consensus.cc:729] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 13842b92ffd84fb6997e79733775466a, State: Initialized, Role: FOLLOWER
Jun 19 12:27:06 dist-test-slave-kr9h krb5kdc[20671](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:27:06.040588 20752 consensus_queue.cc:260] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13842b92ffd84fb6997e79733775466a" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41553 } }
I20260619 12:27:06.040774 20752 raft_consensus.cc:388] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:27:06.040870 20752 raft_consensus.cc:482] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:27:06.040997 20752 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:06.041980 20752 raft_consensus.cc:504] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13842b92ffd84fb6997e79733775466a" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41553 } }
I20260619 12:27:06.042325 20752 leader_election.cc:302] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 13842b92ffd84fb6997e79733775466a; no voters:
I20260619 12:27:06.042673 20752 leader_election.cc:288] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:27:06.042775 20757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:27:06.042997 20757 raft_consensus.cc:686] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [term 1 LEADER]: Becoming Leader. State: Replica: 13842b92ffd84fb6997e79733775466a, State: Running, Role: LEADER
I20260619 12:27:06.043319 20757 consensus_queue.cc:237] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13842b92ffd84fb6997e79733775466a" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41553 } }
I20260619 12:27:06.044097 20752 sys_catalog.cc:564] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:27:06.046496 20755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:06.031884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45478 (local address 127.6.231.254:41553)
0619 12:27:06.032323 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:06.032347 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:06.032394 (+ 47us) server_negotiation.cc:407] Connection header received
0619 12:27:06.033293 (+ 899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:06.033319 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:06.033783 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:06.034091 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:06.034900 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:06.036340 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:06.037211 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:06.037931 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:06.038109 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:06.040837 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:06.040846 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:06.040863 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:06.040900 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:06.043142 (+ 2242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:06.043693 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:06.043699 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:06.043705 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:06.043845 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:06.044171 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:06.044174 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:06.044176 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:06.044662 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:06.044852 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:06.045133 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 12:27:06.045474 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":102,"threads_started":1}
I20260619 12:27:06.046836 20758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "13842b92ffd84fb6997e79733775466a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13842b92ffd84fb6997e79733775466a" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41553 } } }
I20260619 12:27:06.047770 20758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:06.047219 20759 sys_catalog.cc:455] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 13842b92ffd84fb6997e79733775466a. Latest consensus state: current_term: 1 leader_uuid: "13842b92ffd84fb6997e79733775466a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "13842b92ffd84fb6997e79733775466a" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41553 } } }
I20260619 12:27:06.047967 20759 sys_catalog.cc:458] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a [sys.catalog]: This master's current role is: LEADER
W20260619 12:27:06.049297 20772 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:27:06.049386 20772 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:27:06.049494 20774 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:27:06.049531 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:27:06.050297 20774 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:27:06.056114 20774 catalog_manager.cc:1347] Generated new cluster ID: fda28f831fad41eb80860fdca443d6b1
I20260619 12:27:06.056181 20774 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:27:06.078018 20774 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:27:06.079348 20774 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:27:06.091706 20774 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 13842b92ffd84fb6997e79733775466a: Generated new TSK 0
I20260619 12:27:06.092609 20774 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:27:06.106004 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 20684
2026-06-19T12:27:06Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1484 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:27:06 dist-test-slave-kr9h krb5kdc[20779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:27:06 dist-test-slave-kr9h krb5kdc[20779](info): set up 2 sockets
Jun 19 12:27:06 dist-test-slave-kr9h krb5kdc[20779](info): commencing operation
krb5kdc: starting...
W20260619 12:27:07.186118 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:27:07 dist-test-slave-kr9h krb5kdc[20779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:27:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:27:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:27:07.314977 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:33275
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:33275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:27:07.431757 20792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:07.432085 20792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:07.432158 20792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:07.436280 20792 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:27:07.436383 20792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:07.436420 20792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:07.436456 20792 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:27:07.436489 20792 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:27:07.441962 20792 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:33275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:33275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:07.443045 20792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:07.444083 20792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:27:07.452836 20792 server_base.cc:1034] running on GCE node
W20260619 12:27:07.452732 20798 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:07.452796 20797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:07.452732 20800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:07.453660 20792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:07.454782 20792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:07.455984 20792 hybrid_clock.cc:648] HybridClock initialized: now 1781872027455927 us; error 73 us; skew 500 ppm
Jun 19 12:27:07 dist-test-slave-kr9h krb5kdc[20779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872027, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:07.458956 20792 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:27:07.459873 20792 webserver.cc:460] Webserver started at http://127.6.231.254:34459/ using document root <none> and password file <none>
I20260619 12:27:07.460440 20792 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:07.460503 20792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:07.460711 20792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:27:07.462815 20792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "89ea2d09ee034e7db9603dececdcd703"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "08e4ed3a6304a46e180ef8333333e44b"
server_key_iv: "f1d8f19bcd9fd33e15251f0aa8d7cdb1"
server_key_version: "encryptionkey@0"
I20260619 12:27:07.463392 20792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "89ea2d09ee034e7db9603dececdcd703"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "08e4ed3a6304a46e180ef8333333e44b"
server_key_iv: "f1d8f19bcd9fd33e15251f0aa8d7cdb1"
server_key_version: "encryptionkey@0"
I20260619 12:27:07.467053 20792 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.004s
I20260619 12:27:07.469489 20807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:07.470810 20792 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260619 12:27:07.470954 20792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "89ea2d09ee034e7db9603dececdcd703"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "08e4ed3a6304a46e180ef8333333e44b"
server_key_iv: "f1d8f19bcd9fd33e15251f0aa8d7cdb1"
server_key_version: "encryptionkey@0"
I20260619 12:27:07.471067 20792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:07.501015 20792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:07.504189 20792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:07.504411 20792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:07.512035 20792 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:33275
I20260619 12:27:07.512061 20859 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:33275 every 8 connection(s)
I20260619 12:27:07.513006 20792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:27:07.515774 20860 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:07.521302 20860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Bootstrap starting.
I20260619 12:27:07.522131 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 20792
I20260619 12:27:07.522259 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:27:07.522575 7071 external_mini_cluster.cc:1428] Setting key 22cec710492e8e443224d2191919ce61
I20260619 12:27:07.524320 20860 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:07.525069 20860 log.cc:824] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:07.527271 20860 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: No bootstrap required, opened a new log
I20260619 12:27:07.531291 20860 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } }
I20260619 12:27:07.531540 20860 raft_consensus.cc:374] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:07.531622 20860 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ea2d09ee034e7db9603dececdcd703, State: Initialized, Role: FOLLOWER
Jun 19 12:27:07 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872027, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:27:07.532143 20860 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } }
I20260619 12:27:07.532313 20860 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:27:07.532385 20860 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:27:07.532482 20860 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:07.533819 20860 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } }
I20260619 12:27:07.534224 20860 leader_election.cc:302] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89ea2d09ee034e7db9603dececdcd703; no voters:
I20260619 12:27:07.534580 20860 leader_election.cc:288] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:27:07.534721 20865 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:27:07.534955 20865 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 1 LEADER]: Becoming Leader. State: Replica: 89ea2d09ee034e7db9603dececdcd703, State: Running, Role: LEADER
I20260619 12:27:07.535308 20865 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } }
I20260619 12:27:07.535962 20860 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:27:07.536991 20867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89ea2d09ee034e7db9603dececdcd703. Latest consensus state: current_term: 1 leader_uuid: "89ea2d09ee034e7db9603dececdcd703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } } }
I20260619 12:27:07.537113 20867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:07.537530 20866 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "89ea2d09ee034e7db9603dececdcd703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } } }
I20260619 12:27:07.537617 20866 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:07.538250 20874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:27:07.539263 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:07.523815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51616 (local address 127.6.231.254:33275)
0619 12:27:07.524227 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:07.524242 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:07.524275 (+ 33us) server_negotiation.cc:407] Connection header received
0619 12:27:07.525033 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:07.525053 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:07.525518 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:07.525898 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:07.526654 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:07.528090 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:07.528978 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:07.529743 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:07.529936 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:07.532742 (+ 2806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:07.532748 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:07.532765 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:07.532800 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:07.535014 (+ 2214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:07.535551 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:07.535557 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:07.535563 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:07.535697 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:07.536040 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:07.536047 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:07.536050 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:07.536581 (+ 531us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:07.537575 (+ 994us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:07.537749 (+ 174us) server_negotiation.cc:299] Negotiation successful
0619 12:27:07.538067 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":126,"threads_started":1}
I20260619 12:27:07.541254 20874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:27:07.546929 20874 catalog_manager.cc:1347] Generated new cluster ID: 4598321c5c2d4874b741aadb5d7d69b9
I20260619 12:27:07.547024 20874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:27:07.583093 20874 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:27:07.584064 20874 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:27:07.591759 20874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Generated new TSK 0
I20260619 12:27:07.592492 20874 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:27:07.626814 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:33275
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:27:07.742151 20886 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:07.742503 20886 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:07.742610 20886 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:07.747308 20886 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:07.747411 20886 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:07.747514 20886 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:27:07.753774 20886 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20886
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:07.755031 20886 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:07.756203 20886 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:07.767274 20891 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:07.767372 20886 server_base.cc:1034] running on GCE node
W20260619 12:27:07.767274 20892 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:07.767274 20894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:07.767903 20886 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:07.768663 20886 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:07.769861 20886 hybrid_clock.cc:648] HybridClock initialized: now 1781872027769848 us; error 26 us; skew 500 ppm
Jun 19 12:27:07 dist-test-slave-kr9h krb5kdc[20779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872027, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:07.779898 20886 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:27:07.780874 20886 webserver.cc:460] Webserver started at http://127.6.231.193:41073/ using document root <none> and password file <none>
I20260619 12:27:07.781512 20886 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:07.781581 20886 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:07.781767 20886 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:27:07.783826 20886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "7092a48aa9f24f279d983e3867e7a10f"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "48e5170a515a3095cf0e3d59858b9069"
server_key_iv: "b3f29422d400bf9b235504557dc234da"
server_key_version: "encryptionkey@0"
I20260619 12:27:07.784343 20886 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "7092a48aa9f24f279d983e3867e7a10f"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "48e5170a515a3095cf0e3d59858b9069"
server_key_iv: "b3f29422d400bf9b235504557dc234da"
server_key_version: "encryptionkey@0"
I20260619 12:27:07.788146 20886 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 12:27:07.790584 20901 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:07.791838 20886 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:27:07.792004 20886 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "7092a48aa9f24f279d983e3867e7a10f"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "48e5170a515a3095cf0e3d59858b9069"
server_key_iv: "b3f29422d400bf9b235504557dc234da"
server_key_version: "encryptionkey@0"
I20260619 12:27:07.792119 20886 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:07.807667 20886 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:07.811236 20886 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:07.811448 20886 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:07.812048 20886 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:07.812857 20886 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:27:07.812935 20886 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:07.813007 20886 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:27:07.813050 20886 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:07.821875 20886 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:45135
I20260619 12:27:07.821889 21014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:45135 every 8 connection(s)
I20260619 12:27:07.822764 20886 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:27:07.824707 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 20886
I20260619 12:27:07.824836 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:27:07.825155 7071 external_mini_cluster.cc:1428] Setting key 62cf3d207b701abfe5241773afa1ba43
Jun 19 12:27:07 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872027, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:27:07.837466 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:07.824582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:35787 (local address 127.6.231.254:33275)
0619 12:27:07.824719 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:07.824723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:07.825843 (+ 1120us) server_negotiation.cc:407] Connection header received
0619 12:27:07.827358 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:07.827370 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:07.827425 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:07.827500 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:07.828669 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:07.829199 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:07.830449 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:07.830698 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:07.830812 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:07.833574 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:07.833578 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:07.833583 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:07.833613 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:07.835090 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:07.835643 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:07.835650 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:07.835654 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:07.835749 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:07.836136 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:07.836138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:07.836139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:07.836311 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:07.836433 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:07.837101 (+ 668us) server_negotiation.cc:299] Negotiation successful
0619 12:27:07.837247 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:27:07.838476 21017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:07.825120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:35787)
0619 12:27:07.825698 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:07.825758 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:07.827144 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:07.827679 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:07.827686 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:07.828055 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:07.828526 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:07.828538 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:07.829325 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:07.829328 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:07.830310 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:07.830320 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:07.830848 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:07.830852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:07.831046 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:07.831695 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:07.831711 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:07.833377 (+ 1666us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:07.835204 (+ 1827us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:07.835208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:07.835220 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:07.835519 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:07.835831 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:07.835833 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:07.835835 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:07.836047 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:07.836443 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:07.836448 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:07.836790 (+ 342us) client_negotiation.cc:769] Sending connection context
0619 12:27:07.837027 (+ 237us) client_negotiation.cc:240] Negotiation successful
0619 12:27:07.837372 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":132,"threads_started":1}
I20260619 12:27:07.839830 21015 heartbeater.cc:347] Connected to a master server at 127.6.231.254:33275
I20260619 12:27:07.840126 21015 heartbeater.cc:464] Registering TS with master...
I20260619 12:27:07.840624 21015 heartbeater.cc:511] Master 127.6.231.254:33275 requested a full tablet report, sending...
I20260619 12:27:07.842226 20824 ts_manager.cc:194] Registered new tserver with Master: 7092a48aa9f24f279d983e3867e7a10f (127.6.231.193:45135)
I20260619 12:27:07.843992 20824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:35787
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:27:07.856606 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:33275
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:27:07.969048 21020 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:07.969305 21020 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:07.969348 21020 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:07.973410 21020 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:07.973526 21020 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:07.973621 21020 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:27:07.979028 21020 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21020
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:07.979980 21020 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:07.980928 21020 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:07.990782 21026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:07.990774 21025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:07.990852 21020 server_base.cc:1034] running on GCE node
W20260619 12:27:07.990774 21028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:07.991356 21020 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:07.991989 21020 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:07.993183 21020 hybrid_clock.cc:648] HybridClock initialized: now 1781872027993121 us; error 73 us; skew 500 ppm
Jun 19 12:27:07 dist-test-slave-kr9h krb5kdc[20779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872027, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:07.996321 21020 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:27:07.997265 21020 webserver.cc:460] Webserver started at http://127.6.231.194:34859/ using document root <none> and password file <none>
I20260619 12:27:07.997958 21020 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:07.998024 21020 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:07.998252 21020 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:27:08.000432 21020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "a22c80698d9e40dfba1f2c4799afa8ab"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "db4995453b15b86e22e20e7bd4d9a9ff"
server_key_iv: "18ce6cb4e223e916cb19ec560fddd98b"
server_key_version: "encryptionkey@0"
I20260619 12:27:08.001005 21020 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "a22c80698d9e40dfba1f2c4799afa8ab"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "db4995453b15b86e22e20e7bd4d9a9ff"
server_key_iv: "18ce6cb4e223e916cb19ec560fddd98b"
server_key_version: "encryptionkey@0"
I20260619 12:27:08.004926 21020 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 12:27:08.007687 21035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:08.008978 21020 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:27:08.009112 21020 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "a22c80698d9e40dfba1f2c4799afa8ab"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "db4995453b15b86e22e20e7bd4d9a9ff"
server_key_iv: "18ce6cb4e223e916cb19ec560fddd98b"
server_key_version: "encryptionkey@0"
I20260619 12:27:08.009218 21020 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:08.023795 21020 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:08.027189 21020 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:08.027410 21020 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:08.028033 21020 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:08.028903 21020 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:27:08.028952 21020 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:08.028993 21020 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:27:08.029045 21020 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:08.038470 21020 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:37247
I20260619 12:27:08.038544 21148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:37247 every 8 connection(s)
I20260619 12:27:08.039443 21020 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:27:08.043430 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 21020
I20260619 12:27:08.043542 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:27:08.043860 7071 external_mini_cluster.cc:1428] Setting key f163bf6f113f924408c82451fef383d5
Jun 19 12:27:08 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872027, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
I20260619 12:27:08.054086 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.041246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:37033 (local address 127.6.231.254:33275)
0619 12:27:08.041377 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:08.041383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:08.042346 (+ 963us) server_negotiation.cc:407] Connection header received
0619 12:27:08.043866 (+ 1520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:08.043869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:08.043949 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:08.044040 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:08.045340 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.045913 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.047506 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.047746 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.047860 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.050485 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:08.050489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:08.050492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:08.050518 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:08.052007 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:08.052517 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:08.052523 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:08.052524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:08.052614 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:08.052956 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:08.052960 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:08.052962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:08.053148 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:08.053255 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:08.053851 (+ 596us) server_negotiation.cc:299] Negotiation successful
0619 12:27:08.053965 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:27:08.055053 21151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.041752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:37033)
0619 12:27:08.042191 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:08.042233 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:08.043629 (+ 1396us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:08.044269 (+ 640us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:08.044286 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:08.044674 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:08.045199 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.045216 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.046041 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.046048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.047377 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.047388 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.047906 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.047910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.048082 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.048720 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:08.048736 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:08.050318 (+ 1582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:08.052134 (+ 1816us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:08.052140 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:08.052154 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:08.052414 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:08.052706 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:08.052710 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:08.052712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:08.052852 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:08.053255 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:08.053263 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:08.053566 (+ 303us) client_negotiation.cc:769] Sending connection context
0619 12:27:08.053806 (+ 240us) client_negotiation.cc:240] Negotiation successful
0619 12:27:08.054084 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":111,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
I20260619 12:27:08.056241 21149 heartbeater.cc:347] Connected to a master server at 127.6.231.254:33275
I20260619 12:27:08.056514 21149 heartbeater.cc:464] Registering TS with master...
I20260619 12:27:08.057102 21149 heartbeater.cc:511] Master 127.6.231.254:33275 requested a full tablet report, sending...
I20260619 12:27:08.058295 20824 ts_manager.cc:194] Registered new tserver with Master: a22c80698d9e40dfba1f2c4799afa8ab (127.6.231.194:37247)
I20260619 12:27:08.059175 20824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:37033
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:27:08.072013 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:33275
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:27:08.183596 21154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:08.183868 21154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:08.183911 21154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:08.187876 21154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:08.187930 21154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:08.187992 21154 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:27:08.193032 21154 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:08.194053 21154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:08.195011 21154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:08.204640 21160 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:08.204677 21154 server_base.cc:1034] running on GCE node
W20260619 12:27:08.204654 21159 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:08.204759 21162 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:08.205238 21154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:08.205955 21154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:08.207146 21154 hybrid_clock.cc:648] HybridClock initialized: now 1781872028207106 us; error 61 us; skew 500 ppm
Jun 19 12:27:08 dist-test-slave-kr9h krb5kdc[20779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:08.210218 21154 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:27:08.211161 21154 webserver.cc:460] Webserver started at http://127.6.231.195:43387/ using document root <none> and password file <none>
I20260619 12:27:08.211725 21154 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:08.211815 21154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:08.212095 21154 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:27:08.214299 21154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "92faa47eb6f64d278fd8cf7b03d8f948"
format_stamp: "Formatted at 2026-06-19 12:27:08 on dist-test-slave-kr9h"
server_key: "e2e56fe1d39967aa079370e6ce989f47"
server_key_iv: "d8c0ee4ac3853d50e8783ea47f343028"
server_key_version: "encryptionkey@0"
I20260619 12:27:08.214850 21154 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "92faa47eb6f64d278fd8cf7b03d8f948"
format_stamp: "Formatted at 2026-06-19 12:27:08 on dist-test-slave-kr9h"
server_key: "e2e56fe1d39967aa079370e6ce989f47"
server_key_iv: "d8c0ee4ac3853d50e8783ea47f343028"
server_key_version: "encryptionkey@0"
I20260619 12:27:08.218571 21154 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.000s sys 0.004s
I20260619 12:27:08.220965 21169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:08.222230 21154 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:27:08.222373 21154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "92faa47eb6f64d278fd8cf7b03d8f948"
format_stamp: "Formatted at 2026-06-19 12:27:08 on dist-test-slave-kr9h"
server_key: "e2e56fe1d39967aa079370e6ce989f47"
server_key_iv: "d8c0ee4ac3853d50e8783ea47f343028"
server_key_version: "encryptionkey@0"
I20260619 12:27:08.222486 21154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:08.253922 21154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:08.257211 21154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:08.257483 21154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:08.258164 21154 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:08.259032 21154 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:27:08.259104 21154 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:08.259179 21154 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:27:08.259212 21154 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:08.268383 21154 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:35603
I20260619 12:27:08.268407 21282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:35603 every 8 connection(s)
I20260619 12:27:08.269291 21154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:27:08.278610 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 21154
I20260619 12:27:08.278761 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
Jun 19 12:27:08 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:27:08.279130 7071 external_mini_cluster.cc:1428] Setting key c8cf45cbf9b34d802db95acce4b2b56d
I20260619 12:27:08.283785 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.271270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:47331 (local address 127.6.231.254:33275)
0619 12:27:08.271414 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:08.271423 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:08.272237 (+ 814us) server_negotiation.cc:407] Connection header received
0619 12:27:08.273585 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:08.273589 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:08.273649 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:08.273731 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:08.274975 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.275528 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.276685 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.276915 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.277035 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.279741 (+ 2706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:08.279745 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:08.279748 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:08.279780 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:08.281706 (+ 1926us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:08.282234 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:08.282240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:08.282241 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:08.282326 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:08.282671 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:08.282676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:08.282678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:08.282872 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:08.282986 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:08.283498 (+ 512us) server_negotiation.cc:299] Negotiation successful
0619 12:27:08.283619 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:27:08.284633 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.271654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:47331)
0619 12:27:08.272087 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:08.272129 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:08.273294 (+ 1165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:08.273898 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:08.273905 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:08.274272 (+ 367us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:08.274826 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.274853 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.275661 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.275664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.276554 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.276563 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.277074 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.277078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.277248 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.277874 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:08.277890 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:08.279548 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:08.281826 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:08.281832 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:08.281850 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:08.282130 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:08.282419 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:08.282423 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:08.282425 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:08.282576 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:08.282990 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:08.282998 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:08.283258 (+ 260us) client_negotiation.cc:769] Sending connection context
0619 12:27:08.283450 (+ 192us) client_negotiation.cc:240] Negotiation successful
0619 12:27:08.283801 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":101,"threads_started":1}
I20260619 12:27:08.285852 21283 heartbeater.cc:347] Connected to a master server at 127.6.231.254:33275
I20260619 12:27:08.286114 21283 heartbeater.cc:464] Registering TS with master...
I20260619 12:27:08.286633 21283 heartbeater.cc:511] Master 127.6.231.254:33275 requested a full tablet report, sending...
I20260619 12:27:08.287711 20824 ts_manager.cc:194] Registered new tserver with Master: 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603)
I20260619 12:27:08.288295 20824 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:47331
I20260619 12:27:08.293634 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:27:08.303040 20863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.295617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51630 (local address 127.6.231.254:33275)
0619 12:27:08.295784 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:08.295788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:08.296000 (+ 212us) server_negotiation.cc:407] Connection header received
0619 12:27:08.296101 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:08.296103 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:08.296141 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:08.296208 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:08.296989 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.297563 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.298489 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.298689 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.298762 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.300073 (+ 1311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:08.300076 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:08.300078 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:08.300101 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:08.301636 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:08.302005 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:08.302007 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:08.302009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:08.302077 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:08.302301 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:08.302303 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:08.302304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:08.302537 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:08.302639 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:08.302825 (+ 186us) server_negotiation.cc:299] Negotiation successful
0619 12:27:08.302916 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260619 12:27:08.306090 20824 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51630:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:27:08.309113 20824 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:27:08.324244 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.318969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35520 (local address 127.6.231.195:35603)
0619 12:27:08.319395 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:08.319410 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:08.319433 (+ 23us) server_negotiation.cc:407] Connection header received
0619 12:27:08.319505 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:08.319513 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:08.319677 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:08.319833 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:08.320066 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.321203 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.322679 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.323515 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.323626 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.323772 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:08.323968 (+ 196us) server_negotiation.cc:299] Negotiation successful
0619 12:27:08.324068 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":161,"threads_started":1}
I20260619 12:27:08.324220 21296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.318800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:35603 (local address 127.0.0.1:35520)
0619 12:27:08.319050 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:08.319064 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:08.319163 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:08.319813 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:08.319816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:08.319832 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:08.319931 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.319939 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.321355 (+ 1416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.321358 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.322552 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.322560 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.323647 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.323651 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.323763 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.323798 (+ 35us) client_negotiation.cc:769] Sending connection context
0619 12:27:08.323913 (+ 115us) client_negotiation.cc:240] Negotiation successful
0619 12:27:08.324046 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"mutex_wait_us":50,"thread_start_us":111,"threads_started":1}
I20260619 12:27:08.324417 21294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.318041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:45135 (local address 127.0.0.1:48222)
0619 12:27:08.318419 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:08.318461 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:08.318566 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:08.319166 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:08.319175 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:08.319206 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:08.319454 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.319464 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.320979 (+ 1515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.320984 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.322336 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.322346 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.323981 (+ 1635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.323984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.324073 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.324094 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 12:27:08.324166 (+ 72us) client_negotiation.cc:240] Negotiation successful
0619 12:27:08.324234 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":84,"threads_started":1}
I20260619 12:27:08.324273 21298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.319140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:37247 (local address 127.0.0.1:33354)
0619 12:27:08.319369 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:08.319381 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:08.319472 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:08.320577 (+ 1105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:08.320580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:08.320598 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:08.320685 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.320691 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.321898 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.321905 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.322883 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.322894 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.323945 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.323948 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.324024 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.324043 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 12:27:08.324108 (+ 65us) client_negotiation.cc:240] Negotiation successful
0619 12:27:08.324161 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"mutex_wait_us":24,"thread_start_us":82,"threads_started":1}
I20260619 12:27:08.324615 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.319627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33354 (local address 127.6.231.194:37247)
0619 12:27:08.320148 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:08.320160 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:08.320181 (+ 21us) server_negotiation.cc:407] Connection header received
0619 12:27:08.320256 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:08.320260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:08.320430 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:08.320575 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:08.320798 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.321774 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.323019 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.323649 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.323748 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.323885 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:08.324341 (+ 456us) server_negotiation.cc:299] Negotiation successful
0619 12:27:08.324444 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":97,"threads_started":1}
I20260619 12:27:08.324893 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.318194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48222 (local address 127.6.231.193:45135)
0619 12:27:08.318597 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:08.318615 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:08.318641 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:27:08.318738 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:08.318743 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:08.318978 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:08.319625 (+ 647us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:08.319791 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.320860 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.322475 (+ 1615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.323511 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.323626 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.323876 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:08.324598 (+ 722us) server_negotiation.cc:299] Negotiation successful
0619 12:27:08.324716 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":91,"threads_started":1}
I20260619 12:27:08.327523 20949 tablet_service.cc:1467] Processing CreateTablet for tablet 198102bb191a4d69aa6a385bce7a137f (DEFAULT_TABLE table=test-table [id=23221d7e22124cdf848b8a5bb4595f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:08.327539 21217 tablet_service.cc:1467] Processing CreateTablet for tablet 198102bb191a4d69aa6a385bce7a137f (DEFAULT_TABLE table=test-table [id=23221d7e22124cdf848b8a5bb4595f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:08.327539 21083 tablet_service.cc:1467] Processing CreateTablet for tablet 198102bb191a4d69aa6a385bce7a137f (DEFAULT_TABLE table=test-table [id=23221d7e22124cdf848b8a5bb4595f09]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:08.328509 20949 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 198102bb191a4d69aa6a385bce7a137f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:08.328527 21083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 198102bb191a4d69aa6a385bce7a137f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:08.328534 21217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 198102bb191a4d69aa6a385bce7a137f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:08.334791 21300 tablet_bootstrap.cc:492] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Bootstrap starting.
I20260619 12:27:08.336383 21301 tablet_bootstrap.cc:492] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab: Bootstrap starting.
I20260619 12:27:08.337289 21300 tablet_bootstrap.cc:654] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:08.337734 21302 tablet_bootstrap.cc:492] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Bootstrap starting.
I20260619 12:27:08.338827 21300 log.cc:824] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:08.339084 21301 tablet_bootstrap.cc:654] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:08.339855 21301 log.cc:824] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:08.340305 21302 tablet_bootstrap.cc:654] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:08.341027 21302 log.cc:824] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:08.341202 21300 tablet_bootstrap.cc:492] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: No bootstrap required, opened a new log
I20260619 12:27:08.341418 21300 ts_tablet_manager.cc:1397] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260619 12:27:08.341888 21301 tablet_bootstrap.cc:492] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab: No bootstrap required, opened a new log
I20260619 12:27:08.342067 21301 ts_tablet_manager.cc:1397] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260619 12:27:08.342751 21302 tablet_bootstrap.cc:492] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: No bootstrap required, opened a new log
I20260619 12:27:08.342938 21302 ts_tablet_manager.cc:1397] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 12:27:08.345307 21300 raft_consensus.cc:348] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } }
I20260619 12:27:08.345620 21300 raft_consensus.cc:374] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:08.345715 21300 raft_consensus.cc:729] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092a48aa9f24f279d983e3867e7a10f, State: Initialized, Role: FOLLOWER
I20260619 12:27:08.345745 21301 raft_consensus.cc:348] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } }
I20260619 12:27:08.346079 21301 raft_consensus.cc:374] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:08.346141 21301 raft_consensus.cc:729] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a22c80698d9e40dfba1f2c4799afa8ab, State: Initialized, Role: FOLLOWER
I20260619 12:27:08.346204 21300 consensus_queue.cc:260] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } }
I20260619 12:27:08.346210 21302 raft_consensus.cc:348] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } }
I20260619 12:27:08.346486 21302 raft_consensus.cc:374] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:08.346542 21302 raft_consensus.cc:729] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92faa47eb6f64d278fd8cf7b03d8f948, State: Initialized, Role: FOLLOWER
I20260619 12:27:08.346565 21301 consensus_queue.cc:260] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } }
I20260619 12:27:08.346923 21302 consensus_queue.cc:260] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } }
I20260619 12:27:08.347194 21300 ts_tablet_manager.cc:1428] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260619 12:27:08.347210 21149 heartbeater.cc:503] Master 127.6.231.254:33275 was elected leader, sending a full tablet report...
I20260619 12:27:08.347434 21301 ts_tablet_manager.cc:1428] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 12:27:08.347536 21015 heartbeater.cc:503] Master 127.6.231.254:33275 was elected leader, sending a full tablet report...
I20260619 12:27:08.347890 21302 ts_tablet_manager.cc:1428] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260619 12:27:08.348224 21283 heartbeater.cc:503] Master 127.6.231.254:33275 was elected leader, sending a full tablet report...
I20260619 12:27:08.439662 21308 raft_consensus.cc:482] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:27:08.439916 21308 raft_consensus.cc:504] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } }
I20260619 12:27:08.441171 21308 leader_election.cc:288] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7092a48aa9f24f279d983e3867e7a10f (127.6.231.193:45135), a22c80698d9e40dfba1f2c4799afa8ab (127.6.231.194:37247)
I20260619 12:27:08.445633 21285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.441403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:45135 (local address 127.6.231.195:44437)
0619 12:27:08.441560 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:08.441579 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:08.441685 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:08.442130 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:08.442136 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:08.442161 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:08.442298 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.442307 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.443373 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.443376 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.444270 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.444279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.445140 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.445144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.445266 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.445292 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 12:27:08.445382 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 12:27:08.445491 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80,"mutex_wait_us":26}
I20260619 12:27:08.445712 21295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.441708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:44437 (local address 127.6.231.193:45135)
0619 12:27:08.441844 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:08.441849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:08.441867 (+ 18us) server_negotiation.cc:407] Connection header received
0619 12:27:08.441938 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:08.441941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:08.441998 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:08.442096 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:08.442445 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.443249 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.444405 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.444915 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.445012 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.445053 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:08.445491 (+ 438us) server_negotiation.cc:299] Negotiation successful
0619 12:27:08.445572 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 12:27:08.445917 21309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.441749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:37247 (local address 127.6.231.195:39293)
0619 12:27:08.442109 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:08.442121 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:08.442238 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:08.442561 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:08.442563 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:08.442582 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:08.442694 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.442700 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.443632 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.443637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.444530 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.444541 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.445494 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.445499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.445596 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.445619 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 12:27:08.445696 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 12:27:08.445768 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":21,"thread_start_us":123,"threads_started":1}
I20260619 12:27:08.445990 21299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.441917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:39293 (local address 127.6.231.194:37247)
0619 12:27:08.442056 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:08.442061 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:08.442187 (+ 126us) server_negotiation.cc:407] Connection header received
0619 12:27:08.442381 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:08.442387 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:08.442443 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:08.442529 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:08.442834 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.443521 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.444679 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.445284 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.445421 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.445501 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:08.445741 (+ 240us) server_negotiation.cc:299] Negotiation successful
0619 12:27:08.445834 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 12:27:08.446331 20969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "198102bb191a4d69aa6a385bce7a137f" candidate_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092a48aa9f24f279d983e3867e7a10f" is_pre_election: true
I20260619 12:27:08.446698 20969 raft_consensus.cc:2454] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92faa47eb6f64d278fd8cf7b03d8f948 in term 0.
I20260619 12:27:08.446708 21103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "198102bb191a4d69aa6a385bce7a137f" candidate_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" is_pre_election: true
I20260619 12:27:08.446979 21103 raft_consensus.cc:2454] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92faa47eb6f64d278fd8cf7b03d8f948 in term 0.
I20260619 12:27:08.447220 21173 leader_election.cc:302] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092a48aa9f24f279d983e3867e7a10f, 92faa47eb6f64d278fd8cf7b03d8f948; no voters:
I20260619 12:27:08.447604 21308 raft_consensus.cc:2781] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:27:08.447686 21308 raft_consensus.cc:482] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:27:08.447717 21308 raft_consensus.cc:3037] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:08.448777 21308 raft_consensus.cc:504] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } }
I20260619 12:27:08.449131 21308 leader_election.cc:288] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [CANDIDATE]: Term 1 election: Requested vote from peers 7092a48aa9f24f279d983e3867e7a10f (127.6.231.193:45135), a22c80698d9e40dfba1f2c4799afa8ab (127.6.231.194:37247)
I20260619 12:27:08.449714 21103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "198102bb191a4d69aa6a385bce7a137f" candidate_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22c80698d9e40dfba1f2c4799afa8ab"
I20260619 12:27:08.449712 20969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "198102bb191a4d69aa6a385bce7a137f" candidate_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092a48aa9f24f279d983e3867e7a10f"
I20260619 12:27:08.449821 21103 raft_consensus.cc:3037] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:08.449865 20969 raft_consensus.cc:3037] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:08.450937 21103 raft_consensus.cc:2454] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92faa47eb6f64d278fd8cf7b03d8f948 in term 1.
I20260619 12:27:08.451212 20969 raft_consensus.cc:2454] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92faa47eb6f64d278fd8cf7b03d8f948 in term 1.
I20260619 12:27:08.451279 21173 leader_election.cc:302] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92faa47eb6f64d278fd8cf7b03d8f948, a22c80698d9e40dfba1f2c4799afa8ab; no voters:
I20260619 12:27:08.451483 21308 raft_consensus.cc:2781] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:27:08.451713 21308 raft_consensus.cc:686] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 LEADER]: Becoming Leader. State: Replica: 92faa47eb6f64d278fd8cf7b03d8f948, State: Running, Role: LEADER
I20260619 12:27:08.452086 21308 consensus_queue.cc:237] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } }
I20260619 12:27:08.455487 20822 catalog_manager.cc:5495] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195). New cstate: current_term: 1 leader_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } health_report { overall_health: UNKNOWN } } }
I20260619 12:27:08.481786 21297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.477849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35534 (local address 127.6.231.195:35603)
0619 12:27:08.477984 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:08.477987 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:08.478008 (+ 21us) server_negotiation.cc:407] Connection header received
0619 12:27:08.478163 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:08.478166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:08.478221 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:08.478315 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:27:08.478639 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.479153 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.480153 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.480366 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.480445 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.480796 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:27:08.481259 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:27:08.481356 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:08.481538 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 12:27:08.481591 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 12:27:08.487845 21103 raft_consensus.cc:1261] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Refusing update from remote peer 92faa47eb6f64d278fd8cf7b03d8f948: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:27:08.488065 20969 raft_consensus.cc:1261] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Refusing update from remote peer 92faa47eb6f64d278fd8cf7b03d8f948: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:27:08.488597 21308 consensus_queue.cc:1035] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:27:08.488826 21310 consensus_queue.cc:1035] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:27:08.497821 21314 mvcc.cc:205] Tried to move back new op lower bound from 7298547828678946816 to 7298547828542197760. Current Snapshot: MvccSnapshot[applied={T|T < 7298547828678946816}]
I20260619 12:27:08.505942 20822 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:27:08.506100 20822 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51630:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:27:08.508560 20822 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Sending DeleteTablet for 3 replicas of tablet 198102bb191a4d69aa6a385bce7a137f
I20260619 12:27:08.509366 20949 tablet_service.cc:1514] Processing DeleteTablet for tablet 198102bb191a4d69aa6a385bce7a137f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:08 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:48222
I20260619 12:27:08.509469 21083 tablet_service.cc:1514] Processing DeleteTablet for tablet 198102bb191a4d69aa6a385bce7a137f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:08 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:33354
I20260619 12:27:08.509444 21217 tablet_service.cc:1514] Processing DeleteTablet for tablet 198102bb191a4d69aa6a385bce7a137f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:08 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:35520
I20260619 12:27:08.509903 21323 tablet_replica.cc:330] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab: stopping tablet replica
I20260619 12:27:08.510005 21325 tablet_replica.cc:330] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: stopping tablet replica
I20260619 12:27:08.510135 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 20886
I20260619 12:27:08.510221 21323 raft_consensus.cc:2229] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:27:08.510303 21325 raft_consensus.cc:2229] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:27:08.510548 21323 raft_consensus.cc:2258] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:27:08.510771 21325 raft_consensus.cc:2258] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:27:08.512477 21323 ts_tablet_manager.cc:1905] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:27:08.515998 21323 ts_tablet_manager.cc:1918] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:27:08.516170 21323 log.cc:1196] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/198102bb191a4d69aa6a385bce7a137f
I20260619 12:27:08.516564 21323 ts_tablet_manager.cc:1939] T 198102bb191a4d69aa6a385bce7a137f P a22c80698d9e40dfba1f2c4799afa8ab: Deleting consensus metadata
I20260619 12:27:08.518055 20811 catalog_manager.cc:4841] TS a22c80698d9e40dfba1f2c4799afa8ab (127.6.231.194:37247): tablet 198102bb191a4d69aa6a385bce7a137f (table test-table [id=23221d7e22124cdf848b8a5bb4595f09]) successfully deleted
W20260619 12:27:08.519127 20811 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:45135 (error 108)
I20260619 12:27:08.519441 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 21020
I20260619 12:27:08.522298 21298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.522012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:45135 (local address 127.0.0.1:48230)
0619 12:27:08.522150 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:08.522227 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:45135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260619 12:27:08.522533 20811 catalog_manager.cc:4584] TS 7092a48aa9f24f279d983e3867e7a10f (127.6.231.193:45135): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 198102bb191a4d69aa6a385bce7a137f: Network error: Client connection negotiation failed: client connection to 127.6.231.193:45135: connect: Connection refused (error 111)
I20260619 12:27:08.526795 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 21154
I20260619 12:27:08.532702 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 20792
I20260619 12:27:08.538126 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:33275
--webserver_interface=127.6.231.254
--webserver_port=34459
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:33275
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:27:08.653163 21327 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:08.653491 21327 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:08.653600 21327 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:08.657610 21327 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:27:08.657711 21327 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:08.657747 21327 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:08.657783 21327 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:27:08.657817 21327 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:27:08.663368 21327 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:33275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:33275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=34459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21327
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:08.664498 21327 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:08.665592 21327 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:08.678797 21333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:08.678813 21334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:08.678890 21327 server_base.cc:1034] running on GCE node
W20260619 12:27:08.678821 21336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:08.679731 21327 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:08.680874 21327 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:08.682056 21327 hybrid_clock.cc:648] HybridClock initialized: now 1781872028682044 us; error 28 us; skew 500 ppm
Jun 19 12:27:08 dist-test-slave-kr9h krb5kdc[20779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:08.685289 21327 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:27:08.686378 21327 webserver.cc:460] Webserver started at http://127.6.231.254:34459/ using document root <none> and password file <none>
I20260619 12:27:08.686964 21327 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:08.687039 21327 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:08.691138 21327 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.002s
I20260619 12:27:08.693814 21343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:08.695469 21327 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 12:27:08.695601 21327 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "89ea2d09ee034e7db9603dececdcd703"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "08e4ed3a6304a46e180ef8333333e44b"
server_key_iv: "f1d8f19bcd9fd33e15251f0aa8d7cdb1"
server_key_version: "encryptionkey@0"
I20260619 12:27:08.696231 21327 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:08.718605 21327 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:08.721946 21327 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:08.722153 21327 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:08.729403 21327 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:33275
I20260619 12:27:08.729794 21395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:33275 every 8 connection(s)
I20260619 12:27:08.730635 21327 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:27:08.735054 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 21327
I20260619 12:27:08.735405 21396 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:27:08.735555 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:45135
--local_ip_for_outbound_sockets=127.6.231.193
--tserver_master_addrs=127.6.231.254:33275
--webserver_port=41073
--webserver_interface=127.6.231.193
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:27:08.737335 21396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Bootstrap starting.
I20260619 12:27:08.756181 21396 log.cc:824] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:08.765652 21396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:27:08.766072 21396 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Bootstrap complete.
I20260619 12:27:08.771229 21396 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } }
I20260619 12:27:08.771682 21396 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ea2d09ee034e7db9603dececdcd703, State: Initialized, Role: FOLLOWER
I20260619 12:27:08.772213 21396 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } }
I20260619 12:27:08.772370 21396 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:27:08.772471 21396 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:27:08.772606 21396 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:27:08.774801 21396 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } }
I20260619 12:27:08.775178 21396 leader_election.cc:302] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89ea2d09ee034e7db9603dececdcd703; no voters:
I20260619 12:27:08.775557 21396 leader_election.cc:288] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 12:27:08.775648 21400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 12:27:08.775883 21400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 2 LEADER]: Becoming Leader. State: Replica: 89ea2d09ee034e7db9603dececdcd703, State: Running, Role: LEADER
I20260619 12:27:08.776201 21400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } }
I20260619 12:27:08.776891 21396 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:27:08.777222 21400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89ea2d09ee034e7db9603dececdcd703. Latest consensus state: current_term: 2 leader_uuid: "89ea2d09ee034e7db9603dececdcd703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } } }
I20260619 12:27:08.777351 21400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:08.777740 21401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "89ea2d09ee034e7db9603dececdcd703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } } }
I20260619 12:27:08.777949 21401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:08.779124 21404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:27:08.784538 21404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:27:08.785125 21404 catalog_manager.cc:1259] Loaded cluster ID: 4598321c5c2d4874b741aadb5d7d69b9
I20260619 12:27:08.785197 21404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:27:08.786548 21404 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260619 12:27:08.790997 21404 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260619 12:27:08.796648 21404 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:27:08.797505 21404 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Loaded TSK: 0
I20260619 12:27:08.798717 21404 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:27:08.856537 21398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:08.856791 21398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:08.856840 21398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:08.861110 21398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:08.861186 21398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:08.861299 21398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:27:08.866679 21398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:45135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=41073
--tserver_master_addrs=127.6.231.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21398
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:08.867733 21398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:08.868758 21398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:08.879695 21424 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:08.879701 21423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:08.879701 21426 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:08.880107 21398 server_base.cc:1034] running on GCE node
I20260619 12:27:08.880548 21398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:08.881316 21398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:08.882517 21398 hybrid_clock.cc:648] HybridClock initialized: now 1781872028882462 us; error 65 us; skew 500 ppm
Jun 19 12:27:08 dist-test-slave-kr9h krb5kdc[20779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:08.885932 21398 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:27:08.886857 21398 webserver.cc:460] Webserver started at http://127.6.231.193:41073/ using document root <none> and password file <none>
I20260619 12:27:08.887429 21398 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:08.887502 21398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:08.891669 21398 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:27:08.894013 21433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:08.895457 21398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:27:08.895601 21398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "7092a48aa9f24f279d983e3867e7a10f"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "48e5170a515a3095cf0e3d59858b9069"
server_key_iv: "b3f29422d400bf9b235504557dc234da"
server_key_version: "encryptionkey@0"
I20260619 12:27:08.896174 21398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:08.933820 21398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:08.936961 21398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:08.937177 21398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:08.937808 21398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:08.938931 21440 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:27:08.943130 21398 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:27:08.943208 21398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260619 12:27:08.943305 21398 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:27:08.944953 21398 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:27:08.945022 21398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260619 12:27:08.945307 21440 tablet_bootstrap.cc:492] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Bootstrap starting.
I20260619 12:27:08.955566 21398 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:45135
I20260619 12:27:08.955637 21547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:45135 every 8 connection(s)
I20260619 12:27:08.956686 21398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:27:08.962693 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 21398
I20260619 12:27:08.963127 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:37247
--local_ip_for_outbound_sockets=127.6.231.194
--tserver_master_addrs=127.6.231.254:33275
--webserver_port=34859
--webserver_interface=127.6.231.194
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:27:08.965709 21440 log.cc:824] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Log is configured to *not* fsync() on all Append() calls
Jun 19 12:27:08 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:27:08.973889 21440 tablet_bootstrap.cc:492] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:27:08.974313 21440 tablet_bootstrap.cc:492] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Bootstrap complete.
I20260619 12:27:08.975689 21440 ts_tablet_manager.cc:1397] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Time spent bootstrapping tablet: real 0.031s user 0.021s sys 0.007s
I20260619 12:27:08.977046 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.958691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:34323)
0619 12:27:08.959292 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:08.959357 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:08.960564 (+ 1207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:08.961859 (+ 1295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:08.961872 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:08.962409 (+ 537us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:08.963128 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.963168 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.965074 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.965078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.966612 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.966627 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.967614 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.967620 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.967867 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.968660 (+ 793us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:08.968687 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:08.970764 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:08.973382 (+ 2618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:08.973392 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:08.973409 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:08.973824 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:08.974378 (+ 554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:08.974381 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:08.974383 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:08.974546 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:08.975285 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:08.975292 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:08.975585 (+ 293us) client_negotiation.cc:769] Sending connection context
0619 12:27:08.975774 (+ 189us) client_negotiation.cc:240] Negotiation successful
0619 12:27:08.976107 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":157,"threads_started":1}
I20260619 12:27:08.977094 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.958803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:34323 (local address 127.6.231.254:33275)
0619 12:27:08.959676 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:08.959689 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:08.959727 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:27:08.960987 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:08.960999 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:08.961342 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:08.961705 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:08.963381 (+ 1676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.964891 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.966755 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.967448 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.967619 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.970992 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:08.970999 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:08.971015 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:08.971053 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:08.973245 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:08.974036 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:08.974046 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:08.974054 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:08.974233 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:08.974670 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:08.974676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:08.974681 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:08.975161 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:08.975298 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:08.976000 (+ 702us) server_negotiation.cc:299] Negotiation successful
0619 12:27:08.976314 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":762,"thread_start_us":119,"threads_started":1}
I20260619 12:27:08.978768 21440 raft_consensus.cc:348] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } }
I20260619 12:27:08.979266 21440 raft_consensus.cc:729] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092a48aa9f24f279d983e3867e7a10f, State: Initialized, Role: FOLLOWER
I20260619 12:27:08.979794 21440 consensus_queue.cc:260] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } }
I20260619 12:27:08.980098 21548 heartbeater.cc:347] Connected to a master server at 127.6.231.254:33275
I20260619 12:27:08.980332 21548 heartbeater.cc:464] Registering TS with master...
I20260619 12:27:08.980719 21440 ts_tablet_manager.cc:1428] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260619 12:27:08.980844 21548 heartbeater.cc:511] Master 127.6.231.254:33275 requested a full tablet report, sending...
I20260619 12:27:08.983592 21360 ts_manager.cc:194] Registered new tserver with Master: 7092a48aa9f24f279d983e3867e7a10f (127.6.231.193:45135)
I20260619 12:27:08.987613 21360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:34323
I20260619 12:27:08.990286 21548 heartbeater.cc:503] Master 127.6.231.254:33275 was elected leader, sending a full tablet report...
Jun 19 12:27:08 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
I20260619 12:27:08.997231 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.986872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:45135 (local address 127.0.0.1:48246)
0619 12:27:08.987564 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:08.987597 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:08.987757 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:08.988226 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:08.988232 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:08.988711 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:08.988816 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.988822 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.989831 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.989836 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.990515 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:08.990525 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.991221 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.991224 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:08.991343 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.991918 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:08.991932 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:08.993388 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:08.995403 (+ 2015us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:08.995413 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:08.995418 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:08.995715 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:08.996029 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:08.996033 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:08.996035 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:08.996130 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:08.996572 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:08.996578 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:08.996705 (+ 127us) client_negotiation.cc:769] Sending connection context
0619 12:27:08.996844 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 12:27:08.997052 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":531,"thread_start_us":105,"threads_started":1}
I20260619 12:27:08.997267 21556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:08.986918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48246 (local address 127.6.231.193:45135)
0619 12:27:08.987323 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:08.987334 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:08.987692 (+ 358us) server_negotiation.cc:407] Connection header received
0619 12:27:08.987910 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:08.987915 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:08.988067 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:08.988213 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:08.988935 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.989720 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.990636 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:08.991064 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:08.991208 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:08.993565 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:08.993577 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:08.993590 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:08.993621 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:08.995283 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:08.995837 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:08.995849 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:08.995850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:08.995926 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:08.996222 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:08.996226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:08.996228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:08.996452 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:08.996598 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:08.996923 (+ 325us) server_negotiation.cc:299] Negotiation successful
0619 12:27:08.997119 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":113,"threads_started":1}
I20260619 12:27:08.998270 21482 tablet_service.cc:1514] Processing DeleteTablet for tablet 198102bb191a4d69aa6a385bce7a137f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:08 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:48246
I20260619 12:27:08.999269 21557 tablet_replica.cc:330] stopping tablet replica
I20260619 12:27:08.999748 21557 raft_consensus.cc:2229] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:27:09.000062 21557 raft_consensus.cc:2258] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:27:09.001137 21557 ts_tablet_manager.cc:1905] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:27:09.004709 21557 ts_tablet_manager.cc:1918] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:27:09.004824 21557 log.cc:1196] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal/wals/198102bb191a4d69aa6a385bce7a137f
I20260619 12:27:09.005090 21557 ts_tablet_manager.cc:1939] T 198102bb191a4d69aa6a385bce7a137f P 7092a48aa9f24f279d983e3867e7a10f: Deleting consensus metadata
I20260619 12:27:09.006808 21347 catalog_manager.cc:4841] TS 7092a48aa9f24f279d983e3867e7a10f (127.6.231.193:45135): tablet 198102bb191a4d69aa6a385bce7a137f (table test-table [id=23221d7e22124cdf848b8a5bb4595f09]) successfully deleted
W20260619 12:27:09.086319 21552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:09.086614 21552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:09.086659 21552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:09.091027 21552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:09.091095 21552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:09.091192 21552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:27:09.096817 21552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:37247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=34859
--tserver_master_addrs=127.6.231.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:09.098016 21552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:09.099053 21552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:27:09.108985 21552 server_base.cc:1034] running on GCE node
W20260619 12:27:09.108948 21563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:09.108947 21562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:09.108960 21565 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:09.109627 21552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:09.110353 21552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:09.111541 21552 hybrid_clock.cc:648] HybridClock initialized: now 1781872029111524 us; error 32 us; skew 500 ppm
Jun 19 12:27:09 dist-test-slave-kr9h krb5kdc[20779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872029, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:09.114375 21552 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:27:09.115295 21552 webserver.cc:460] Webserver started at http://127.6.231.194:34859/ using document root <none> and password file <none>
I20260619 12:27:09.115859 21552 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:09.115973 21552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:09.120405 21552 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:27:09.122537 21572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:09.123745 21552 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:27:09.123878 21552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "a22c80698d9e40dfba1f2c4799afa8ab"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "db4995453b15b86e22e20e7bd4d9a9ff"
server_key_iv: "18ce6cb4e223e916cb19ec560fddd98b"
server_key_version: "encryptionkey@0"
I20260619 12:27:09.124440 21552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:09.145031 21552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:09.148563 21552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:09.148777 21552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:09.149394 21552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:09.150295 21552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:27:09.150364 21552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:09.150436 21552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:27:09.150501 21552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:09.159749 21552 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:37247
I20260619 12:27:09.159771 21685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:37247 every 8 connection(s)
I20260619 12:27:09.160784 21552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:27:09.170519 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 21552
I20260619 12:27:09.170925 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:35603
--local_ip_for_outbound_sockets=127.6.231.195
--tserver_master_addrs=127.6.231.254:33275
--webserver_port=43387
--webserver_interface=127.6.231.195
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Jun 19 12:27:09 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872029, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:27:09.176127 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.162834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:55873 (local address 127.6.231.254:33275)
0619 12:27:09.162982 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:09.162986 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:09.163818 (+ 832us) server_negotiation.cc:407] Connection header received
0619 12:27:09.165128 (+ 1310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:09.165131 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:09.165189 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:09.165271 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:09.166476 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.167053 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.168203 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.168451 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.168565 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.171961 (+ 3396us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:09.171964 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:09.171966 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:09.171992 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:09.173677 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:09.174294 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:09.174297 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:09.174298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:09.174371 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:09.174791 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:09.174800 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:09.174803 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:09.175013 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:09.175105 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:09.175783 (+ 678us) server_negotiation.cc:299] Negotiation successful
0619 12:27:09.175898 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 12:27:09.177380 21688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.163222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:55873)
0619 12:27:09.163662 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:09.163706 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:09.164918 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:09.165479 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:09.165488 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:09.165852 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:09.166323 (+ 471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:09.166336 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.167189 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.167195 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:09.168068 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:09.168077 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.168607 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.168611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:09.168817 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.169500 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:09.169516 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:09.171764 (+ 2248us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:09.173789 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:09.173798 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:09.173817 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:09.174189 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:09.174465 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:09.174471 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:09.174476 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:09.174673 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:09.175144 (+ 471us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:09.175153 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:09.175513 (+ 360us) client_negotiation.cc:769] Sending connection context
0619 12:27:09.175750 (+ 237us) client_negotiation.cc:240] Negotiation successful
0619 12:27:09.176155 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":112,"threads_started":1}
I20260619 12:27:09.178746 21686 heartbeater.cc:347] Connected to a master server at 127.6.231.254:33275
I20260619 12:27:09.179023 21686 heartbeater.cc:464] Registering TS with master...
I20260619 12:27:09.179669 21686 heartbeater.cc:511] Master 127.6.231.254:33275 requested a full tablet report, sending...
I20260619 12:27:09.180891 21360 ts_manager.cc:194] Registered new tserver with Master: a22c80698d9e40dfba1f2c4799afa8ab (127.6.231.194:37247)
I20260619 12:27:09.181630 21360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:55873
W20260619 12:27:09.288932 21689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:09.289228 21689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:09.289310 21689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:09.293372 21689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:09.293478 21689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:09.293596 21689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:27:09.298820 21689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:35603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=43387
--tserver_master_addrs=127.6.231.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21689
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:09.299963 21689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:09.301079 21689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:09.310752 21697 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:09.310791 21694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:09.310873 21689 server_base.cc:1034] running on GCE node
W20260619 12:27:09.310767 21695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:09.311448 21689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:09.312134 21689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:09.313302 21689 hybrid_clock.cc:648] HybridClock initialized: now 1781872029313282 us; error 31 us; skew 500 ppm
Jun 19 12:27:09 dist-test-slave-kr9h krb5kdc[20779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872029, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:09.316165 21689 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:27:09.316982 21689 webserver.cc:460] Webserver started at http://127.6.231.195:43387/ using document root <none> and password file <none>
I20260619 12:27:09.317610 21689 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:09.317688 21689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:09.321715 21689 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:27:09.323815 21704 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:09.325270 21689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:27:09.325399 21689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "92faa47eb6f64d278fd8cf7b03d8f948"
format_stamp: "Formatted at 2026-06-19 12:27:08 on dist-test-slave-kr9h"
server_key: "e2e56fe1d39967aa079370e6ce989f47"
server_key_iv: "d8c0ee4ac3853d50e8783ea47f343028"
server_key_version: "encryptionkey@0"
I20260619 12:27:09.326028 21689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:09.343946 21689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:09.346920 21689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:09.347136 21689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:09.347749 21689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:09.348901 21711 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:27:09.353029 21689 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:27:09.353099 21689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260619 12:27:09.353178 21689 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:27:09.355014 21689 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:27:09.355063 21689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260619 12:27:09.355473 21711 tablet_bootstrap.cc:492] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Bootstrap starting.
I20260619 12:27:09.365383 21689 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:35603
I20260619 12:27:09.365504 21818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:35603 every 8 connection(s)
I20260619 12:27:09.366390 21689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:27:09.368407 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 21689
I20260619 12:27:09.374970 21711 log.cc:824] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Log is configured to *not* fsync() on all Append() calls
Jun 19 12:27:09 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872029, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:27:09.379601 21823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.369487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59656 (local address 127.6.231.254:33275)
0619 12:27:09.369771 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:09.369776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:09.369942 (+ 166us) server_negotiation.cc:407] Connection header received
0619 12:27:09.370082 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:09.370085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:09.370136 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:09.370243 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:09.371365 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.372073 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.373034 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.373394 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.373525 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.375255 (+ 1730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:09.375262 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:09.375265 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:09.375296 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:09.377018 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:09.377882 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:09.377887 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:09.377889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:09.377986 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:09.378619 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:09.378622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:09.378625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:09.378854 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:09.379137 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:09.379290 (+ 153us) server_negotiation.cc:299] Negotiation successful
0619 12:27:09.379419 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"spinlock_wait_cycles":18176,"thread_start_us":109,"threads_started":1}
I20260619 12:27:09.381855 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.368254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:43241 (local address 127.6.231.254:33275)
0619 12:27:09.368401 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:09.368405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:09.369058 (+ 653us) server_negotiation.cc:407] Connection header received
0619 12:27:09.370275 (+ 1217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:09.370278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:09.370333 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:09.370379 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:09.371814 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.372631 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.374122 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.374341 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.374447 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.377332 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:09.377336 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:09.377339 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:09.377367 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:09.378920 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:09.380100 (+ 1180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:09.380103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:09.380105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:09.380188 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:09.380738 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:09.380741 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:09.380743 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:09.380913 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:09.381017 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:09.381645 (+ 628us) server_negotiation.cc:299] Negotiation successful
0619 12:27:09.381743 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62,"spinlock_wait_cycles":3712}
I20260619 12:27:09.383085 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.368365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:43241)
0619 12:27:09.368894 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:09.368946 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:09.370058 (+ 1112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:09.370693 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:09.370699 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:09.371034 (+ 335us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:09.371647 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:09.371667 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.372793 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.372797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:09.373999 (+ 1202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:09.374009 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.374482 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.374486 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:09.374659 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.375343 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:09.375365 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:09.377183 (+ 1818us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:09.379593 (+ 2410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:09.379599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:09.379615 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:09.379968 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:09.380276 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:09.380280 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:09.380282 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:09.380622 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:09.381028 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:09.381037 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:09.381316 (+ 279us) client_negotiation.cc:769] Sending connection context
0619 12:27:09.382087 (+ 771us) client_negotiation.cc:240] Negotiation successful
0619 12:27:09.382343 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":156,"threads_started":1}
I20260619 12:27:09.383637 21711 tablet_bootstrap.cc:492] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:27:09.384020 21711 tablet_bootstrap.cc:492] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Bootstrap complete.
I20260619 12:27:09.384150 21819 heartbeater.cc:347] Connected to a master server at 127.6.231.254:33275
I20260619 12:27:09.384357 21819 heartbeater.cc:464] Registering TS with master...
I20260619 12:27:09.384775 21819 heartbeater.cc:511] Master 127.6.231.254:33275 requested a full tablet report, sending...
I20260619 12:27:09.385180 21711 ts_tablet_manager.cc:1397] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Time spent bootstrapping tablet: real 0.030s user 0.018s sys 0.008s
I20260619 12:27:09.386431 21360 ts_manager.cc:194] Registered new tserver with Master: 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603)
I20260619 12:27:09.387677 21360 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:43241
I20260619 12:27:09.388235 21711 raft_consensus.cc:348] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } }
I20260619 12:27:09.388859 21711 raft_consensus.cc:729] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92faa47eb6f64d278fd8cf7b03d8f948, State: Initialized, Role: FOLLOWER
I20260619 12:27:09.389336 21711 consensus_queue.cc:260] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } }
I20260619 12:27:09.389989 21819 heartbeater.cc:503] Master 127.6.231.254:33275 was elected leader, sending a full tablet report...
I20260619 12:27:09.390421 21711 ts_tablet_manager.cc:1428] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Time spent starting tablet: real 0.005s user 0.005s sys 0.003s
I20260619 12:27:09.393379 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Jun 19 12:27:09 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872028, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.195@KRBTEST.COM
I20260619 12:27:09.397850 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.387417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:35603 (local address 127.0.0.1:40778)
0619 12:27:09.387584 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:09.387596 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:09.387688 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:09.388334 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:09.388337 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:09.388526 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:09.388627 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:09.388635 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.390011 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.390014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:09.390816 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:09.390826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.391570 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.391574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:09.391787 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.392379 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:09.392392 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:09.394130 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:09.396220 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:09.396223 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:09.396226 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:09.396426 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:09.396726 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:09.396730 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:09.396732 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:09.396818 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:09.397273 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:09.397276 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:09.397371 (+ 95us) client_negotiation.cc:769] Sending connection context
0619 12:27:09.397525 (+ 154us) client_negotiation.cc:240] Negotiation successful
0619 12:27:09.397679 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110}
I20260619 12:27:09.397924 21825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.387496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40778 (local address 127.6.231.195:35603)
0619 12:27:09.387848 (+ 352us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:09.387859 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:09.387905 (+ 46us) server_negotiation.cc:407] Connection header received
0619 12:27:09.387974 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:09.387979 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:09.388184 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:09.388320 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:09.388764 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.389892 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.390925 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.391455 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.391538 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.394286 (+ 2748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:09.394294 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:09.394310 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:09.394342 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:09.396088 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:09.396537 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:09.396544 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:09.396546 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:09.396632 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:09.396914 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:09.396917 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:09.396919 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:09.397168 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:09.397310 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:09.397608 (+ 298us) server_negotiation.cc:299] Negotiation successful
0619 12:27:09.397767 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":128,"threads_started":1}
I20260619 12:27:09.398119 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.393993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59666 (local address 127.6.231.254:33275)
0619 12:27:09.394129 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:09.394133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:09.394389 (+ 256us) server_negotiation.cc:407] Connection header received
0619 12:27:09.394546 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:09.394548 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:09.394639 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:09.394755 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:27:09.394988 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.395582 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.396470 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.396730 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.396842 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.397174 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:27:09.397605 (+ 431us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:27:09.397729 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:09.397788 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 12:27:09.397837 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:27:09.399174 21753 tablet_service.cc:1514] Processing DeleteTablet for tablet 198102bb191a4d69aa6a385bce7a137f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:08 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:40778
I20260619 12:27:09.400167 21828 tablet_replica.cc:330] stopping tablet replica
I20260619 12:27:09.400485 21828 raft_consensus.cc:2229] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:27:09.400759 21828 raft_consensus.cc:2258] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:27:09.399041 21358 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:59666:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260619 12:27:09.401870 21828 ts_tablet_manager.cc:1905] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 12:27:09.403209 21358 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:27:09.406327 21828 ts_tablet_manager.cc:1918] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:27:09.406411 21828 log.cc:1196] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/198102bb191a4d69aa6a385bce7a137f
I20260619 12:27:09.406688 21828 ts_tablet_manager.cc:1939] T 198102bb191a4d69aa6a385bce7a137f P 92faa47eb6f64d278fd8cf7b03d8f948: Deleting consensus metadata
I20260619 12:27:09.409083 21347 catalog_manager.cc:4841] TS 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603): tablet 198102bb191a4d69aa6a385bce7a137f (table test-table [id=23221d7e22124cdf848b8a5bb4595f09]) successfully deleted
I20260619 12:27:09.411914 21753 tablet_service.cc:1467] Processing CreateTablet for tablet 88a517fd16b645aa91f7cf75d7242847 (DEFAULT_TABLE table=test-table [id=01c2d8822236484bbf26323aade57f39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:09.411911 21482 tablet_service.cc:1467] Processing CreateTablet for tablet 88a517fd16b645aa91f7cf75d7242847 (DEFAULT_TABLE table=test-table [id=01c2d8822236484bbf26323aade57f39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:09.412523 21753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a517fd16b645aa91f7cf75d7242847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:09.412529 21482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a517fd16b645aa91f7cf75d7242847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:09.415089 21711 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Bootstrap starting.
I20260619 12:27:09.415228 21440 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f: Bootstrap starting.
I20260619 12:27:09.415884 21711 tablet_bootstrap.cc:654] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:09.416059 21440 tablet_bootstrap.cc:654] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:09.416797 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.410994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:37247 (local address 127.0.0.1:42888)
0619 12:27:09.411120 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:09.411137 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:09.411221 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:09.412097 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:09.412100 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:09.412119 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:09.412237 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:09.412248 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.413731 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.413734 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:09.414962 (+ 1228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:09.414976 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.416256 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.416260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:09.416392 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.416451 (+ 59us) client_negotiation.cc:769] Sending connection context
0619 12:27:09.416549 (+ 98us) client_negotiation.cc:240] Negotiation successful
0619 12:27:09.416645 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 12:27:09.416842 21832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.411153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42888 (local address 127.6.231.194:37247)
0619 12:27:09.411552 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:09.411566 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:09.411592 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:27:09.411696 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:09.411702 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:09.411925 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:09.412226 (+ 301us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:09.412422 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.413536 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.415114 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.416077 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.416243 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.416434 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:09.416580 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 12:27:09.416693 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":113,"threads_started":1}
I20260619 12:27:09.417778 21440 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f: No bootstrap required, opened a new log
I20260619 12:27:09.417907 21440 ts_tablet_manager.cc:1397] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260619 12:27:09.418202 21711 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: No bootstrap required, opened a new log
I20260619 12:27:09.418290 21711 ts_tablet_manager.cc:1397] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260619 12:27:09.418397 21440 raft_consensus.cc:348] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:09.418519 21440 raft_consensus.cc:374] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:09.418565 21440 raft_consensus.cc:729] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092a48aa9f24f279d983e3867e7a10f, State: Initialized, Role: FOLLOWER
I20260619 12:27:09.418707 21440 consensus_queue.cc:260] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:09.418732 21711 raft_consensus.cc:348] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:09.418828 21711 raft_consensus.cc:374] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:09.418871 21711 raft_consensus.cc:729] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92faa47eb6f64d278fd8cf7b03d8f948, State: Initialized, Role: FOLLOWER
I20260619 12:27:09.418987 21440 ts_tablet_manager.cc:1428] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:27:09.418983 21711 consensus_queue.cc:260] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:09.419241 21711 ts_tablet_manager.cc:1428] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:27:09.420598 21620 tablet_service.cc:1467] Processing CreateTablet for tablet 88a517fd16b645aa91f7cf75d7242847 (DEFAULT_TABLE table=test-table [id=01c2d8822236484bbf26323aade57f39]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:09.421741 21620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a517fd16b645aa91f7cf75d7242847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:09.428953 21835 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab: Bootstrap starting.
I20260619 12:27:09.431586 21835 tablet_bootstrap.cc:654] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:09.432348 21835 log.cc:824] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:09.434221 21835 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab: No bootstrap required, opened a new log
I20260619 12:27:09.434428 21835 ts_tablet_manager.cc:1397] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 12:27:09.437530 21835 raft_consensus.cc:348] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:09.437763 21835 raft_consensus.cc:374] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:09.437836 21835 raft_consensus.cc:729] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a22c80698d9e40dfba1f2c4799afa8ab, State: Initialized, Role: FOLLOWER
I20260619 12:27:09.438299 21835 consensus_queue.cc:260] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:09.439190 21686 heartbeater.cc:503] Master 127.6.231.254:33275 was elected leader, sending a full tablet report...
I20260619 12:27:09.439253 21835 ts_tablet_manager.cc:1428] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260619 12:27:09.458034 21549 tablet.cc:2369] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:27:09.617537 21820 tablet.cc:2369] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:27:09.644425 21826 raft_consensus.cc:482] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:27:09.644696 21826 raft_consensus.cc:504] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:09.646050 21826 leader_election.cc:288] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a22c80698d9e40dfba1f2c4799afa8ab (127.6.231.194:37247), 7092a48aa9f24f279d983e3867e7a10f (127.6.231.193:45135)
I20260619 12:27:09.651384 21839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.646797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:40833 (local address 127.6.231.193:45135)
0619 12:27:09.647038 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:09.647042 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:09.647057 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:27:09.647157 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:09.647163 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:09.647230 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:09.647371 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:09.647634 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.648302 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.649846 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.650539 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.650663 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.650847 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:09.651173 (+ 326us) server_negotiation.cc:299] Negotiation successful
0619 12:27:09.651248 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":102,"threads_started":1}
I20260619 12:27:09.651382 21832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.646459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:50687 (local address 127.6.231.194:37247)
0619 12:27:09.646587 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:09.646607 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:09.646629 (+ 22us) server_negotiation.cc:407] Connection header received
0619 12:27:09.647025 (+ 396us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:09.647029 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:09.647091 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:09.647190 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:09.647481 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.648131 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.649846 (+ 1715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.650517 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.650663 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.650717 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:09.651173 (+ 456us) server_negotiation.cc:299] Negotiation successful
0619 12:27:09.651255 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:27:09.651386 21821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.646339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:37247 (local address 127.6.231.195:50687)
0619 12:27:09.646481 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:09.646498 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:09.646612 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:09.647209 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:09.647211 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:09.647221 (+ 10us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:09.647341 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:09.647349 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.648304 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.648307 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:09.649714 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:09.649725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.650661 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.650665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:09.650959 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.651060 (+ 101us) client_negotiation.cc:769] Sending connection context
0619 12:27:09.651142 (+ 82us) client_negotiation.cc:240] Negotiation successful
0619 12:27:09.651223 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":111}
I20260619 12:27:09.651392 21838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.646649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:45135 (local address 127.6.231.195:40833)
0619 12:27:09.646997 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:09.647008 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:09.647080 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:09.647370 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:09.647375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:09.647395 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:09.647516 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:09.647522 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.648402 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.648405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:09.649714 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:09.649725 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.650661 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.650665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:09.650963 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.651061 (+ 98us) client_negotiation.cc:769] Sending connection context
0619 12:27:09.651142 (+ 81us) client_negotiation.cc:240] Negotiation successful
0619 12:27:09.651223 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":142,"thread_start_us":146,"threads_started":1}
I20260619 12:27:09.652302 21495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a517fd16b645aa91f7cf75d7242847" candidate_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092a48aa9f24f279d983e3867e7a10f" is_pre_election: true
I20260619 12:27:09.652341 21640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a517fd16b645aa91f7cf75d7242847" candidate_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" is_pre_election: true
I20260619 12:27:09.652617 21640 raft_consensus.cc:2454] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92faa47eb6f64d278fd8cf7b03d8f948 in term 0.
I20260619 12:27:09.652612 21495 raft_consensus.cc:2454] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92faa47eb6f64d278fd8cf7b03d8f948 in term 0.
I20260619 12:27:09.653133 21708 leader_election.cc:302] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092a48aa9f24f279d983e3867e7a10f, 92faa47eb6f64d278fd8cf7b03d8f948; no voters:
I20260619 12:27:09.653507 21826 raft_consensus.cc:2781] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:27:09.653606 21826 raft_consensus.cc:482] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:27:09.653849 21826 raft_consensus.cc:3037] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:09.655051 21826 raft_consensus.cc:504] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:09.655459 21826 leader_election.cc:288] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [CANDIDATE]: Term 1 election: Requested vote from peers a22c80698d9e40dfba1f2c4799afa8ab (127.6.231.194:37247), 7092a48aa9f24f279d983e3867e7a10f (127.6.231.193:45135)
I20260619 12:27:09.655947 21640 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a517fd16b645aa91f7cf75d7242847" candidate_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a22c80698d9e40dfba1f2c4799afa8ab"
I20260619 12:27:09.656118 21495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a517fd16b645aa91f7cf75d7242847" candidate_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7092a48aa9f24f279d983e3867e7a10f"
I20260619 12:27:09.656119 21640 raft_consensus.cc:3037] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:09.656267 21495 raft_consensus.cc:3037] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:09.657603 21495 raft_consensus.cc:2454] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92faa47eb6f64d278fd8cf7b03d8f948 in term 1.
I20260619 12:27:09.657604 21640 raft_consensus.cc:2454] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92faa47eb6f64d278fd8cf7b03d8f948 in term 1.
I20260619 12:27:09.657915 21708 leader_election.cc:302] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92faa47eb6f64d278fd8cf7b03d8f948, a22c80698d9e40dfba1f2c4799afa8ab; no voters:
I20260619 12:27:09.658102 21826 raft_consensus.cc:2781] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:27:09.658353 21826 raft_consensus.cc:686] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 LEADER]: Becoming Leader. State: Replica: 92faa47eb6f64d278fd8cf7b03d8f948, State: Running, Role: LEADER
I20260619 12:27:09.658638 21826 consensus_queue.cc:237] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
W20260619 12:27:09.661939 21687 tablet.cc:2369] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:27:09.661804 21358 catalog_manager.cc:5495] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195). New cstate: current_term: 1 leader_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } health_report { overall_health: UNKNOWN } } }
I20260619 12:27:09.714226 21825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.709904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40782 (local address 127.6.231.195:35603)
0619 12:27:09.710075 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:09.710079 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:09.710094 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:27:09.710171 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:09.710179 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:09.710236 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:09.710333 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:27:09.710651 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.711609 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.712549 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:09.712832 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:09.712919 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:09.713253 (+ 334us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:27:09.713837 (+ 584us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:27:09.713931 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:09.714050 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 12:27:09.714101 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 12:27:09.718271 21640 raft_consensus.cc:1261] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Refusing update from remote peer 92faa47eb6f64d278fd8cf7b03d8f948: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:27:09.718267 21495 raft_consensus.cc:1261] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Refusing update from remote peer 92faa47eb6f64d278fd8cf7b03d8f948: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:27:09.719151 21826 consensus_queue.cc:1035] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:27:09.719383 21840 consensus_queue.cc:1035] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:27:09.728047 21844 mvcc.cc:205] Tried to move back new op lower bound from 7298547833718697984 to 7298547833484468224. Current Snapshot: MvccSnapshot[applied={T|T < 7298547833718697984}]
I20260619 12:27:09.730643 21843 mvcc.cc:205] Tried to move back new op lower bound from 7298547833718697984 to 7298547833484468224. Current Snapshot: MvccSnapshot[applied={T|T < 7298547833718697984 or (T in {7298547833718697984})}]
I20260619 12:27:09.738926 21360 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:59666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:27:09.739110 21360 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:59666:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:27:09.741803 21360 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Sending DeleteTablet for 3 replicas of tablet 88a517fd16b645aa91f7cf75d7242847
I20260619 12:27:09.742591 21753 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a517fd16b645aa91f7cf75d7242847 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:09 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:40778
I20260619 12:27:09.742635 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 21398
I20260619 12:27:09.742815 21482 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a517fd16b645aa91f7cf75d7242847 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:09 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:48246
I20260619 12:27:09.743149 21620 tablet_service.cc:1514] Processing DeleteTablet for tablet 88a517fd16b645aa91f7cf75d7242847 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:09 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:42888
I20260619 12:27:09.743396 21828 tablet_replica.cc:330] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: stopping tablet replica
I20260619 12:27:09.743551 21828 raft_consensus.cc:2229] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:27:09.743913 21828 raft_consensus.cc:2258] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:27:09.744894 21828 ts_tablet_manager.cc:1905] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:27:09.747591 21828 ts_tablet_manager.cc:1918] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:27:09.747668 21828 log.cc:1196] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/88a517fd16b645aa91f7cf75d7242847
I20260619 12:27:09.747934 21828 ts_tablet_manager.cc:1939] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Deleting consensus metadata
I20260619 12:27:09.749004 21347 catalog_manager.cc:4841] TS 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603): tablet 88a517fd16b645aa91f7cf75d7242847 (table test-table [id=01c2d8822236484bbf26323aade57f39]) successfully deleted
I20260619 12:27:09.749913 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 21552
W20260619 12:27:09.749953 21347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:45135 (error 108)
I20260619 12:27:09.752040 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.751668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:45135 (local address 127.0.0.1:35970)
0619 12:27:09.751879 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:09.751951 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:45135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132}
W20260619 12:27:09.752312 21347 catalog_manager.cc:4584] TS 7092a48aa9f24f279d983e3867e7a10f (127.6.231.193:45135): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 88a517fd16b645aa91f7cf75d7242847: Network error: Client connection negotiation failed: client connection to 127.6.231.193:45135: connect: Connection refused (error 111)
I20260619 12:27:09.758396 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 21689
I20260619 12:27:09.759763 21555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:09.759484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:37247 (local address 127.0.0.1:42900)
0619 12:27:09.759619 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:09.759681 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.194:37247: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260619 12:27:09.764372 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 21327
I20260619 12:27:09.770691 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:33275
--webserver_interface=127.6.231.254
--webserver_port=34459
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:33275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:27:09.884443 21854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:09.884797 21854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:09.884887 21854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:09.889065 21854 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:27:09.889168 21854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:09.889204 21854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:09.889242 21854 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:27:09.889281 21854 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:27:09.895067 21854 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:33275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:33275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=34459
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:09.896246 21854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:09.897303 21854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:09.906215 21860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:09.906369 21862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:09.906484 21854 server_base.cc:1034] running on GCE node
W20260619 12:27:09.906215 21859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:09.907138 21854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:09.908311 21854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:09.909519 21854 hybrid_clock.cc:648] HybridClock initialized: now 1781872029909491 us; error 44 us; skew 500 ppm
Jun 19 12:27:09 dist-test-slave-kr9h krb5kdc[20779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872029, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:09.912422 21854 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:27:09.913518 21854 webserver.cc:460] Webserver started at http://127.6.231.254:34459/ using document root <none> and password file <none>
I20260619 12:27:09.914094 21854 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:09.914189 21854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:09.918573 21854 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:27:09.920816 21869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:09.922227 21854 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:27:09.922384 21854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "89ea2d09ee034e7db9603dececdcd703"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "08e4ed3a6304a46e180ef8333333e44b"
server_key_iv: "f1d8f19bcd9fd33e15251f0aa8d7cdb1"
server_key_version: "encryptionkey@0"
I20260619 12:27:09.922972 21854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:09.947309 21854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:09.950929 21854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:09.951148 21854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:09.958694 21854 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:33275
I20260619 12:27:09.959012 21921 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:33275 every 8 connection(s)
I20260619 12:27:09.959832 21854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:27:09.964149 21922 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:27:09.965927 21922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Bootstrap starting.
I20260619 12:27:09.967511 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 21854
I20260619 12:27:09.967963 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:45135
--local_ip_for_outbound_sockets=127.6.231.193
--tserver_master_addrs=127.6.231.254:33275
--webserver_port=41073
--webserver_interface=127.6.231.193
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:27:09.983877 21922 log.cc:824] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:09.995473 21922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:27:09.995867 21922 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: Bootstrap complete.
I20260619 12:27:09.999925 21922 raft_consensus.cc:348] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } }
I20260619 12:27:10.000316 21922 raft_consensus.cc:729] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 89ea2d09ee034e7db9603dececdcd703, State: Initialized, Role: FOLLOWER
I20260619 12:27:10.000869 21922 consensus_queue.cc:260] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } }
I20260619 12:27:10.000991 21922 raft_consensus.cc:388] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:27:10.001075 21922 raft_consensus.cc:482] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:27:10.001189 21922 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 2 FOLLOWER]: Advancing to term 3
I20260619 12:27:10.003224 21922 raft_consensus.cc:504] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } }
I20260619 12:27:10.003562 21922 leader_election.cc:302] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 89ea2d09ee034e7db9603dececdcd703; no voters:
I20260619 12:27:10.003906 21922 leader_election.cc:288] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 12:27:10.003988 21926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 12:27:10.004168 21926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [term 3 LEADER]: Becoming Leader. State: Replica: 89ea2d09ee034e7db9603dececdcd703, State: Running, Role: LEADER
I20260619 12:27:10.004467 21926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } }
I20260619 12:27:10.005182 21922 sys_catalog.cc:564] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:27:10.006119 21927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "89ea2d09ee034e7db9603dececdcd703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } } }
I20260619 12:27:10.006245 21927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:10.006170 21928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 89ea2d09ee034e7db9603dececdcd703. Latest consensus state: current_term: 3 leader_uuid: "89ea2d09ee034e7db9603dececdcd703" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "89ea2d09ee034e7db9603dececdcd703" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33275 } } }
I20260619 12:27:10.006296 21928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703 [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:10.010675 21942 catalog_manager.cc:1259] Loaded cluster ID: 4598321c5c2d4874b741aadb5d7d69b9
I20260619 12:27:10.010746 21942 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: loading cluster ID for follower catalog manager: success
W20260619 12:27:10.011793 21942 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 89ea2d09ee034e7db9603dececdcd703: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260619 12:27:10.011873 21942 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260619 12:27:10.011950 21934 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:27:10.014254 21934 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:27:10.014684 21934 catalog_manager.cc:1259] Loaded cluster ID: 4598321c5c2d4874b741aadb5d7d69b9
I20260619 12:27:10.014758 21934 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260619 12:27:10.015262 21934 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f759dbf99ad google::LogMessage::Fail() at ??:0
@ 0x7f759dbfc88a google::LogMessage::SendToLog() at ??:0
@ 0x7f759dbf94de google::LogMessage::Flush() at ??:0
@ 0x7f759dbfaa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f75a8538fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f75a8539d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f75a8536579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f75a857a640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f759fe4b3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f759fe4bc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f759fe4d3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f759fe3c512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f75a14806db start_thread at ??:0
@ 0x7f759c6f761f clone at ??:0
W20260619 12:27:10.110136 21924 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:10.110437 21924 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:10.110481 21924 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:10.114820 21924 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:10.114886 21924 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:10.114981 21924 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:27:10.120373 21924 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:45135
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=41073
--tserver_master_addrs=127.6.231.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21924
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:10.121542 21924 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:10.122604 21924 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:10.132766 21964 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:10.132658 21963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:10.132874 21924 server_base.cc:1034] running on GCE node
W20260619 12:27:10.132658 21966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:10.133579 21924 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:10.134351 21924 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:10.135526 21924 hybrid_clock.cc:648] HybridClock initialized: now 1781872030135499 us; error 50 us; skew 500 ppm
Jun 19 12:27:10 dist-test-slave-kr9h krb5kdc[20779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:10.138387 21924 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:27:10.139424 21924 webserver.cc:460] Webserver started at http://127.6.231.193:41073/ using document root <none> and password file <none>
I20260619 12:27:10.139989 21924 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:10.140051 21924 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:10.144160 21924 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:27:10.146422 21973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:10.147758 21924 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:27:10.147852 21924 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "7092a48aa9f24f279d983e3867e7a10f"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "48e5170a515a3095cf0e3d59858b9069"
server_key_iv: "b3f29422d400bf9b235504557dc234da"
server_key_version: "encryptionkey@0"
I20260619 12:27:10.148419 21924 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:10.159070 21924 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:10.162432 21924 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:10.162643 21924 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:10.163295 21924 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:10.164393 21980 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:27:10.168509 21924 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:27:10.168574 21924 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260619 12:27:10.168620 21924 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:27:10.170414 21924 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:27:10.170473 21924 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260619 12:27:10.170744 21980 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f: Bootstrap starting.
I20260619 12:27:10.181458 21924 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:45135
I20260619 12:27:10.181675 22087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:45135 every 8 connection(s)
I20260619 12:27:10.182755 21924 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:27:10.185164 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 21924
I20260619 12:27:10.185716 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:37247
--local_ip_for_outbound_sockets=127.6.231.194
--tserver_master_addrs=127.6.231.254:33275
--webserver_port=34859
--webserver_interface=127.6.231.194
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:27:10.192539 21980 log.cc:824] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:10.202792 21980 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:27:10.203269 21980 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f: Bootstrap complete.
I20260619 12:27:10.204751 21980 ts_tablet_manager.cc:1397] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f: Time spent bootstrapping tablet: real 0.034s user 0.021s sys 0.009s
I20260619 12:27:10.208350 21980 raft_consensus.cc:348] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:10.208943 21980 raft_consensus.cc:729] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7092a48aa9f24f279d983e3867e7a10f, State: Initialized, Role: FOLLOWER
I20260619 12:27:10.209535 21980 consensus_queue.cc:260] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:10.210645 21980 ts_tablet_manager.cc:1428] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260619 12:27:10.216609 22090 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:27:10.185082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:46305)
0619 12:27:10.186467 (+ 1385us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:10.186529 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:10.187775 (+ 1246us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:10.215849 (+ 28074us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":1145,"thread_start_us":194,"threads_started":1}
W20260619 12:27:10.217203 22088 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:33275 (0 consecutive failures): Network error: Failed to ping master at 127.6.231.254:33275: Client connection negotiation failed: client connection to 127.6.231.254:33275: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 12:27:10.217978 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:10.217691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:39287)
0619 12:27:10.217814 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:10.217885 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 12:27:10.218717 22090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:10.218468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:35749)
0619 12:27:10.218590 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:10.218649 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 12:27:10.218962 22088 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:27:10.309630 22091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:10.309935 22091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:10.310026 22091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:10.314368 22091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:10.314435 22091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:10.314517 22091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:27:10.320034 22091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:37247
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=34859
--tserver_master_addrs=127.6.231.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:10.321245 22091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:10.322386 22091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:10.332592 22098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:10.332829 22091 server_base.cc:1034] running on GCE node
W20260619 12:27:10.332597 22101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:10.332672 22099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:10.333478 22091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:10.334215 22091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:10.335386 22091 hybrid_clock.cc:648] HybridClock initialized: now 1781872030335366 us; error 32 us; skew 500 ppm
Jun 19 12:27:10 dist-test-slave-kr9h krb5kdc[20779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:10.338440 22091 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:27:10.339359 22091 webserver.cc:460] Webserver started at http://127.6.231.194:34859/ using document root <none> and password file <none>
I20260619 12:27:10.339941 22091 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:10.340005 22091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:10.344091 22091 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:27:10.346328 22108 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:10.347741 22091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:27:10.347985 22091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "a22c80698d9e40dfba1f2c4799afa8ab"
format_stamp: "Formatted at 2026-06-19 12:27:07 on dist-test-slave-kr9h"
server_key: "db4995453b15b86e22e20e7bd4d9a9ff"
server_key_iv: "18ce6cb4e223e916cb19ec560fddd98b"
server_key_version: "encryptionkey@0"
I20260619 12:27:10.348583 22091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:10.368371 22091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:10.371940 22091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:10.372177 22091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:10.372813 22091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:10.373989 22115 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:27:10.378190 22091 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:27:10.378319 22091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260619 12:27:10.378405 22091 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:27:10.380131 22091 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:27:10.380203 22091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260619 12:27:10.380450 22115 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab: Bootstrap starting.
I20260619 12:27:10.391198 22091 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:37247
I20260619 12:27:10.391283 22222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:37247 every 8 connection(s)
I20260619 12:27:10.392284 22091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:27:10.394889 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 22091
I20260619 12:27:10.395359 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:35603
--local_ip_for_outbound_sockets=127.6.231.195
--tserver_master_addrs=127.6.231.254:33275
--webserver_port=43387
--webserver_interface=127.6.231.195
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:38481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:27:10.396364 22225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:10.394629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:46345)
0619 12:27:10.395224 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:10.395380 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":154,"threads_started":1}
W20260619 12:27:10.397327 22223 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:33275 (0 consecutive failures): Network error: Failed to ping master at 127.6.231.254:33275: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
I20260619 12:27:10.398185 22225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:10.397875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:36075)
0619 12:27:10.398031 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:10.398101 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 12:27:10.398865 22225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:10.398648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:35599)
0619 12:27:10.398765 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:10.398816 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 12:27:10.399025 22223 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:27:10.400487 22115 log.cc:824] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:10.410789 22115 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:27:10.411244 22115 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab: Bootstrap complete.
I20260619 12:27:10.412751 22115 ts_tablet_manager.cc:1397] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab: Time spent bootstrapping tablet: real 0.033s user 0.025s sys 0.004s
I20260619 12:27:10.416585 22115 raft_consensus.cc:348] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:10.417225 22115 raft_consensus.cc:729] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a22c80698d9e40dfba1f2c4799afa8ab, State: Initialized, Role: FOLLOWER
I20260619 12:27:10.417805 22115 consensus_queue.cc:260] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:10.419034 22115 ts_tablet_manager.cc:1428] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab: Time spent starting tablet: real 0.006s user 0.009s sys 0.000s
I20260619 12:27:10.419234 22225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:10.418935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:50569)
0619 12:27:10.419066 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:10.419140 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 12:27:10.419910 22225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:10.419688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:60359)
0619 12:27:10.419801 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:10.419849 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 12:27:10.517668 22226 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:10.517937 22226 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:10.517985 22226 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:10.522109 22226 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:10.522177 22226 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:10.522269 22226 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:27:10.527720 22226 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:35603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=43387
--tserver_master_addrs=127.6.231.254:33275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22226
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:10.529101 22226 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:10.530218 22226 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:10.540344 22236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:10.540458 22233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:10.540426 22226 server_base.cc:1034] running on GCE node
W20260619 12:27:10.540361 22234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:10.541040 22226 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:10.541770 22226 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:10.542934 22226 hybrid_clock.cc:648] HybridClock initialized: now 1781872030542902 us; error 53 us; skew 500 ppm
Jun 19 12:27:10 dist-test-slave-kr9h krb5kdc[20779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:10.545939 22226 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:27:10.546784 22226 webserver.cc:460] Webserver started at http://127.6.231.195:43387/ using document root <none> and password file <none>
I20260619 12:27:10.547381 22226 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:10.547452 22226 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:10.551403 22226 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:27:10.553977 22243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:10.555341 22226 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:27:10.555473 22226 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "92faa47eb6f64d278fd8cf7b03d8f948"
format_stamp: "Formatted at 2026-06-19 12:27:08 on dist-test-slave-kr9h"
server_key: "e2e56fe1d39967aa079370e6ce989f47"
server_key_iv: "d8c0ee4ac3853d50e8783ea47f343028"
server_key_version: "encryptionkey@0"
I20260619 12:27:10.556085 22226 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:10.580422 22226 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:10.583998 22226 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:10.584250 22226 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:10.584911 22226 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:10.585875 22226 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:27:10.585952 22226 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:10.586026 22226 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:27:10.586071 22226 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:10.595487 22226 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:35603
I20260619 12:27:10.595503 22356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:35603 every 8 connection(s)
I20260619 12:27:10.596566 22226 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:27:10.600546 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:10.598861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:35429)
0619 12:27:10.599367 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:10.599796 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":126,"threads_started":1}
W20260619 12:27:10.601099 22357 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:33275 (0 consecutive failures): Network error: Failed to ping master at 127.6.231.254:33275: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
I20260619 12:27:10.601800 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:10.601577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:48321)
0619 12:27:10.601684 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:10.601736 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 12:27:10.602460 22359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:10.602261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:56021)
0619 12:27:10.602360 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:10.602405 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 12:27:10.602627 22357 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:27:10.605726 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 22226
I20260619 12:27:11.220166 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:11.219692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:40589)
0619 12:27:11.219993 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:11.220066 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":85,"threads_started":1}
I20260619 12:27:11.421281 22362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:11.420768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:41847)
0619 12:27:11.421117 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:11.421184 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":97,"threads_started":1}
I20260619 12:27:11.544447 22363 raft_consensus.cc:482] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:27:11.544684 22363 raft_consensus.cc:504] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:11.546008 22363 leader_election.cc:288] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers a22c80698d9e40dfba1f2c4799afa8ab (127.6.231.194:37247), 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603)
I20260619 12:27:11.550113 22367 raft_consensus.cc:482] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:27:11.550346 22367 raft_consensus.cc:504] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:11.551349 22367 leader_election.cc:288] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603), 7092a48aa9f24f279d983e3867e7a10f (127.6.231.193:45135)
Jun 19 12:27:11 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.195@KRBTEST.COM
Jun 19 12:27:11 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.194@KRBTEST.COM
Jun 19 12:27:11 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
Jun 19 12:27:11 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.195@KRBTEST.COM
I20260619 12:27:11.562657 22361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:11.546244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:37247 (local address 127.6.231.193:44701)
0619 12:27:11.546376 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:11.546391 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:11.546505 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:11.547849 (+ 1344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:11.547856 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:11.548352 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:11.548812 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:11.548826 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.550235 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.550238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:11.551622 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:11.551633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.552849 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.552856 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:11.553033 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:11.554228 (+ 1195us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:11.554246 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:11.556976 (+ 2730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:11.559715 (+ 2739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:11.559724 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:11.559740 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:11.560147 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:11.560543 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:11.560549 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:11.560553 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:11.560737 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:11.561637 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:11.561646 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:11.561918 (+ 272us) client_negotiation.cc:769] Sending connection context
0619 12:27:11.562123 (+ 205us) client_negotiation.cc:240] Negotiation successful
0619 12:27:11.562478 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":62}
I20260619 12:27:11.562673 22364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:11.546391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:44701 (local address 127.6.231.194:37247)
0619 12:27:11.546817 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:11.546831 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:11.546880 (+ 49us) server_negotiation.cc:407] Connection header received
0619 12:27:11.547149 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:11.547155 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:11.547502 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:11.547814 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:11.548972 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.550124 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.552001 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.552717 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.552877 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:11.557137 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:11.557144 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:11.557157 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:11.557188 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:11.559579 (+ 2391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:11.560261 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:11.560268 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:11.560275 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:11.560417 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:11.560862 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:11.560868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:11.560869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:11.561423 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:11.561630 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:11.562229 (+ 599us) server_negotiation.cc:299] Negotiation successful
0619 12:27:11.562478 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":101,"threads_started":1}
I20260619 12:27:11.564467 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a517fd16b645aa91f7cf75d7242847" candidate_uuid: "7092a48aa9f24f279d983e3867e7a10f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" is_pre_election: true
I20260619 12:27:11.564630 22369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:11.551787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:45135 (local address 127.6.231.194:44157)
0619 12:27:11.552354 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:11.552368 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:11.552447 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:11.553726 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:11.553732 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:11.553946 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:11.554051 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:11.554059 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.555405 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.555407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:11.555932 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:11.555940 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.557107 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.557110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:11.557235 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:11.557841 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:11.557860 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:11.559693 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:11.562802 (+ 3109us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:11.562813 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:11.562820 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:11.563116 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:11.563486 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:11.563489 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:11.563491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:11.563585 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:11.564093 (+ 508us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:11.564098 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:11.564210 (+ 112us) client_negotiation.cc:769] Sending connection context
0619 12:27:11.564323 (+ 113us) client_negotiation.cc:240] Negotiation successful
0619 12:27:11.564474 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":5,"thread_start_us":50,"threads_started":1}
I20260619 12:27:11.564759 22177 raft_consensus.cc:2454] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092a48aa9f24f279d983e3867e7a10f in term 1.
I20260619 12:27:11.565191 22370 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:11.552035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:44157 (local address 127.6.231.193:45135)
0619 12:27:11.553238 (+ 1203us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:11.553244 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:11.553265 (+ 21us) server_negotiation.cc:407] Connection header received
0619 12:27:11.553307 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:11.553312 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:11.553525 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:11.553736 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:11.554192 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.555284 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.556145 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.556956 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.557543 (+ 587us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:11.559814 (+ 2271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:11.559823 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:11.559834 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:11.559862 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:11.562672 (+ 2810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:11.563261 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:11.563269 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:11.563272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:11.563365 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:11.563697 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:11.563700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:11.563702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:11.563957 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:11.564800 (+ 843us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:11.564902 (+ 102us) server_negotiation.cc:299] Negotiation successful
0619 12:27:11.565037 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1098,"thread_start_us":75,"threads_started":1}
I20260619 12:27:11.565635 22362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:11.551520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:35603 (local address 127.6.231.194:44227)
0619 12:27:11.551668 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:11.551690 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:11.551813 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:11.552941 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:11.552946 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:11.553420 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:11.553571 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:11.553578 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.554530 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.554534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:11.555522 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:11.555542 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.557137 (+ 1595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.557141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:11.557422 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:11.558187 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:11.558202 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:11.560249 (+ 2047us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:11.563596 (+ 3347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:11.563602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:11.563606 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:11.563827 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:11.564235 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:11.564238 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:11.564246 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:11.564325 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:11.565137 (+ 812us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:11.565141 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:11.565234 (+ 93us) client_negotiation.cc:769] Sending connection context
0619 12:27:11.565318 (+ 84us) client_negotiation.cc:240] Negotiation successful
0619 12:27:11.565484 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":210}
I20260619 12:27:11.565693 22365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:11.546508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:35603 (local address 127.6.231.193:37107)
0619 12:27:11.546834 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:11.546848 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:11.546939 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:11.548229 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:11.548232 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:11.548433 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:11.548810 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:11.548825 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.550152 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.550156 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:11.551832 (+ 1676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:11.551842 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.553148 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.553151 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:11.553243 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:11.554022 (+ 779us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:11.554044 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:11.556956 (+ 2912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:11.563556 (+ 6600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:11.563562 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:11.563568 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:11.563809 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:11.564222 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:11.564225 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:11.564227 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:11.564314 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:11.565180 (+ 866us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:11.565183 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:11.565275 (+ 92us) client_negotiation.cc:769] Sending connection context
0619 12:27:11.565378 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 12:27:11.565526 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":88,"spinlock_wait_cycles":14592,"thread_start_us":92,"threads_started":1}
I20260619 12:27:11.566497 22368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:11.551700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:44227 (local address 127.6.231.195:35603)
0619 12:27:11.552682 (+ 982us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:11.552693 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:11.552706 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:27:11.552741 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:11.552746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:11.552797 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:11.553484 (+ 687us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:11.553705 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.554401 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.556639 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.556998 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.557228 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:11.560386 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:11.560393 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:11.560397 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:11.560426 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:11.562673 (+ 2247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:11.563939 (+ 1266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:11.563944 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:11.563947 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:11.564110 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:11.564432 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:11.564435 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:11.564437 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:11.565014 (+ 577us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:11.565786 (+ 772us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:11.565934 (+ 148us) server_negotiation.cc:299] Negotiation successful
0619 12:27:11.566303 (+ 369us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":107,"server-negotiator.queue_time_us":906,"spinlock_wait_cycles":36224,"thread_start_us":89,"threads_started":1}
I20260619 12:27:11.566497 22366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:11.546692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:37107 (local address 127.6.231.195:35603)
0619 12:27:11.547004 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:11.547020 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:11.547058 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:27:11.547671 (+ 613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:11.547676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:11.547975 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:11.548285 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:11.548980 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.550005 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.551950 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:11.552791 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:11.552985 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:11.557160 (+ 4175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:11.557166 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:11.557181 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:11.557214 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:11.563461 (+ 6247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:11.563937 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:11.563942 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:11.563947 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:11.564110 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:11.564421 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:11.564426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:11.564428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:11.565014 (+ 586us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:11.565135 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:11.565934 (+ 799us) server_negotiation.cc:299] Negotiation successful
0619 12:27:11.566357 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":188,"server-negotiator.queue_time_us":206,"spinlock_wait_cycles":22016,"thread_start_us":135,"threads_started":1}
I20260619 12:27:11.566746 21977 leader_election.cc:302] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092a48aa9f24f279d983e3867e7a10f, a22c80698d9e40dfba1f2c4799afa8ab; no voters:
I20260619 12:27:11.566764 22042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a517fd16b645aa91f7cf75d7242847" candidate_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7092a48aa9f24f279d983e3867e7a10f" is_pre_election: true
I20260619 12:27:11.567063 22042 raft_consensus.cc:2454] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a22c80698d9e40dfba1f2c4799afa8ab in term 1.
I20260619 12:27:11.567162 22363 raft_consensus.cc:2781] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260619 12:27:11.567234 22363 raft_consensus.cc:482] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:27:11.567274 22363 raft_consensus.cc:3037] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:27:11.568557 22112 leader_election.cc:302] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7092a48aa9f24f279d983e3867e7a10f, a22c80698d9e40dfba1f2c4799afa8ab; no voters:
I20260619 12:27:11.568822 22367 raft_consensus.cc:2781] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Leader pre-election won for term 2
W20260619 12:27:11.568819 21977 leader_election.cc:341] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603): Not found: Tablet not found: 88a517fd16b645aa91f7cf75d7242847
I20260619 12:27:11.568897 22367 raft_consensus.cc:482] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:27:11.568936 22367 raft_consensus.cc:3037] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:27:11.570813 22363 raft_consensus.cc:504] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:11.571210 22363 leader_election.cc:288] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [CANDIDATE]: Term 2 election: Requested vote from peers a22c80698d9e40dfba1f2c4799afa8ab (127.6.231.194:37247), 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603)
I20260619 12:27:11.571218 22367 raft_consensus.cc:504] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:11.571619 22367 leader_election.cc:288] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [CANDIDATE]: Term 2 election: Requested vote from peers 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603), 7092a48aa9f24f279d983e3867e7a10f (127.6.231.193:45135)
W20260619 12:27:11.568943 22112 leader_election.cc:341] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603): Not found: Tablet not found: 88a517fd16b645aa91f7cf75d7242847
W20260619 12:27:11.571818 21977 leader_election.cc:341] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603): Not found: Tablet not found: 88a517fd16b645aa91f7cf75d7242847
I20260619 12:27:11.572393 22042 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a517fd16b645aa91f7cf75d7242847" candidate_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "7092a48aa9f24f279d983e3867e7a10f"
I20260619 12:27:11.572399 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a517fd16b645aa91f7cf75d7242847" candidate_uuid: "7092a48aa9f24f279d983e3867e7a10f" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a22c80698d9e40dfba1f2c4799afa8ab"
I20260619 12:27:11.572568 22042 raft_consensus.cc:2379] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate a22c80698d9e40dfba1f2c4799afa8ab in current term 2: Already voted for candidate 7092a48aa9f24f279d983e3867e7a10f in this term.
I20260619 12:27:11.572613 22177 raft_consensus.cc:2379] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 7092a48aa9f24f279d983e3867e7a10f in current term 2: Already voted for candidate a22c80698d9e40dfba1f2c4799afa8ab in this term.
W20260619 12:27:11.572670 22112 leader_election.cc:341] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603): Not found: Tablet not found: 88a517fd16b645aa91f7cf75d7242847
I20260619 12:27:11.573011 21977 leader_election.cc:302] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 7092a48aa9f24f279d983e3867e7a10f; no voters: 92faa47eb6f64d278fd8cf7b03d8f948, a22c80698d9e40dfba1f2c4799afa8ab
I20260619 12:27:11.573043 22112 leader_election.cc:302] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a22c80698d9e40dfba1f2c4799afa8ab; no voters: 7092a48aa9f24f279d983e3867e7a10f, 92faa47eb6f64d278fd8cf7b03d8f948
I20260619 12:27:11.573207 22363 raft_consensus.cc:2726] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 12:27:11.573230 22367 raft_consensus.cc:2726] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260619 12:27:11.603566 22371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:11.603142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:45715)
0619 12:27:11.603428 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:11.603480 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":106,"threads_started":1}
I20260619 12:27:12.221403 22372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:12.220958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:52583)
0619 12:27:12.221250 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:12.221308 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":80,"threads_started":1}
I20260619 12:27:12.422502 22373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:12.422081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:51041)
0619 12:27:12.422346 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:12.422407 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":81,"threads_started":1}
I20260619 12:27:12.604780 22374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:12.604328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:44551)
0619 12:27:12.604613 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:12.604680 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":73,"threads_started":1}
I20260619 12:27:13.095189 22375 raft_consensus.cc:482] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:27:13.095391 22375 raft_consensus.cc:504] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 2 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:13.095880 22375 leader_election.cc:288] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers a22c80698d9e40dfba1f2c4799afa8ab (127.6.231.194:37247), 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603)
I20260619 12:27:13.096380 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a517fd16b645aa91f7cf75d7242847" candidate_uuid: "7092a48aa9f24f279d983e3867e7a10f" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" is_pre_election: true
I20260619 12:27:13.096568 22177 raft_consensus.cc:2454] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7092a48aa9f24f279d983e3867e7a10f in term 2.
W20260619 12:27:13.096894 21977 leader_election.cc:341] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [CANDIDATE]: Term 3 pre-election: Tablet error from VoteRequest() call to peer 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603): Not found: Tablet not found: 88a517fd16b645aa91f7cf75d7242847
I20260619 12:27:13.097083 21977 leader_election.cc:302] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7092a48aa9f24f279d983e3867e7a10f, a22c80698d9e40dfba1f2c4799afa8ab; no voters: 92faa47eb6f64d278fd8cf7b03d8f948
I20260619 12:27:13.097364 22375 raft_consensus.cc:2781] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260619 12:27:13.097467 22375 raft_consensus.cc:482] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:27:13.097524 22375 raft_consensus.cc:3037] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 2 FOLLOWER]: Advancing to term 3
I20260619 12:27:13.100651 22375 raft_consensus.cc:504] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:13.101037 22375 leader_election.cc:288] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [CANDIDATE]: Term 3 election: Requested vote from peers a22c80698d9e40dfba1f2c4799afa8ab (127.6.231.194:37247), 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603)
I20260619 12:27:13.101531 22177 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "88a517fd16b645aa91f7cf75d7242847" candidate_uuid: "7092a48aa9f24f279d983e3867e7a10f" candidate_term: 3 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "a22c80698d9e40dfba1f2c4799afa8ab"
I20260619 12:27:13.101672 22177 raft_consensus.cc:3037] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 2 FOLLOWER]: Advancing to term 3
W20260619 12:27:13.101879 21977 leader_election.cc:341] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [CANDIDATE]: Term 3 election: Tablet error from VoteRequest() call to peer 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603): Not found: Tablet not found: 88a517fd16b645aa91f7cf75d7242847
I20260619 12:27:13.102680 22177 raft_consensus.cc:2454] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7092a48aa9f24f279d983e3867e7a10f in term 3.
I20260619 12:27:13.102991 21977 leader_election.cc:302] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 7092a48aa9f24f279d983e3867e7a10f, a22c80698d9e40dfba1f2c4799afa8ab; no voters: 92faa47eb6f64d278fd8cf7b03d8f948
I20260619 12:27:13.103173 22375 raft_consensus.cc:2781] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 3 FOLLOWER]: Leader election won for term 3
I20260619 12:27:13.103369 22375 raft_consensus.cc:686] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [term 3 LEADER]: Becoming Leader. State: Replica: 7092a48aa9f24f279d983e3867e7a10f, State: Running, Role: LEADER
I20260619 12:27:13.103817 22375 consensus_queue.cc:237] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:13.104158 22377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:13.103775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:59911)
0619 12:27:13.104025 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:13.104082 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":61,"threads_started":1}
I20260619 12:27:13.424005 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:13.423476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:49317)
0619 12:27:13.423807 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:13.423883 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":78,"threads_started":1}
I20260619 12:27:13.606140 22380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:13.605731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:46441)
0619 12:27:13.605996 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:13.606052 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":85,"threads_started":1}
W20260619 12:27:13.637181 21977 consensus_peers.cc:487] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f -> Peer 92faa47eb6f64d278fd8cf7b03d8f948 (127.6.231.195:35603): Couldn't send request to peer 92faa47eb6f64d278fd8cf7b03d8f948. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 88a517fd16b645aa91f7cf75d7242847. This is attempt 1: this message will repeat every 5th retry.
I20260619 12:27:13.700733 22177 raft_consensus.cc:1261] T 88a517fd16b645aa91f7cf75d7242847 P a22c80698d9e40dfba1f2c4799afa8ab [term 3 FOLLOWER]: Refusing update from remote peer 7092a48aa9f24f279d983e3867e7a10f: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 3 index: 3. (index mismatch)
I20260619 12:27:13.701490 22381 consensus_queue.cc:1035] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260619 12:27:13.703473 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:13.703222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:55767)
0619 12:27:13.703337 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:13.703395 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260619 12:27:13.706374 22387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:13.705941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:54177)
0619 12:27:13.706209 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:13.706270 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":106,"threads_started":1}
I20260619 12:27:13.706593 22379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:13.706229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:41723)
0619 12:27:13.706379 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:13.706506 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260619 12:27:14.179100 22388 ts_tablet_manager.cc:927] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Initiating tablet copy from peer 7092a48aa9f24f279d983e3867e7a10f (127.6.231.193:45135)
I20260619 12:27:14.179701 22388 tablet_copy_client.cc:316] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: tablet copy: Beginning tablet copy session from remote peer at address 127.6.231.193:45135
Jun 19 12:27:14 dist-test-slave-kr9h krb5kdc[20779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872030, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
I20260619 12:27:14.190156 22390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:14.180265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:45135 (local address 127.6.231.195:44585)
0619 12:27:14.180544 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:14.180656 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:14.180827 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:14.181163 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:14.181170 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:14.181710 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:14.181837 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:14.181843 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:14.182655 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:14.182658 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:14.183333 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:14.183341 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:14.184034 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:14.184038 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:14.184164 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:14.184781 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:14.184794 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:14.186241 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:14.188129 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:14.188137 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:14.188142 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:14.188388 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:14.188727 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:14.188729 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:14.188731 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:14.188819 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:14.189291 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:14.189297 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:14.189415 (+ 118us) client_negotiation.cc:769] Sending connection context
0619 12:27:14.189875 (+ 460us) client_negotiation.cc:240] Negotiation successful
0619 12:27:14.190026 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":78,"threads_started":1}
I20260619 12:27:14.190224 22391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:14.180410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:44585 (local address 127.6.231.193:45135)
0619 12:27:14.180820 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:14.180839 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:14.180858 (+ 19us) server_negotiation.cc:407] Connection header received
0619 12:27:14.180939 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:14.180942 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:14.180998 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:14.181119 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:14.181964 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:14.182493 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:14.183531 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:14.183766 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:14.184028 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:14.186393 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:14.186396 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:14.186398 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:14.186420 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:14.188019 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:14.188506 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:14.188509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:14.188510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:14.188594 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:14.188951 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:14.188955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:14.188956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:14.189171 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:14.189259 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:14.189994 (+ 735us) server_negotiation.cc:299] Negotiation successful
0619 12:27:14.190104 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":131,"threads_started":1}
I20260619 12:27:14.190858 22062 tablet_copy_service.cc:140] P 7092a48aa9f24f279d983e3867e7a10f: Received BeginTabletCopySession request for tablet 88a517fd16b645aa91f7cf75d7242847 from peer 92faa47eb6f64d278fd8cf7b03d8f948 ({username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:44585)
I20260619 12:27:14.190994 22062 tablet_copy_service.cc:161] P 7092a48aa9f24f279d983e3867e7a10f: Beginning new tablet copy session on tablet 88a517fd16b645aa91f7cf75d7242847 from peer 92faa47eb6f64d278fd8cf7b03d8f948 at {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:44585: session id = 92faa47eb6f64d278fd8cf7b03d8f948-88a517fd16b645aa91f7cf75d7242847
I20260619 12:27:14.192170 22062 tablet_copy_source_session.cc:205] T 88a517fd16b645aa91f7cf75d7242847 P 7092a48aa9f24f279d983e3867e7a10f: Tablet Copy: opened 0 blocks and 1 log segments
I20260619 12:27:14.196079 22388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 88a517fd16b645aa91f7cf75d7242847. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:14.202814 22388 tablet_copy_client.cc:791] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: tablet copy: Starting download of 0 data blocks...
I20260619 12:27:14.203043 22388 tablet_copy_client.cc:655] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: tablet copy: Starting download of 1 WAL segments...
I20260619 12:27:14.204923 22388 tablet_copy_client.cc:531] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260619 12:27:14.206969 22388 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Bootstrap starting.
I20260619 12:27:14.212600 22388 log.cc:824] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:14.219426 22388 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:27:14.219835 22388 tablet_bootstrap.cc:492] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Bootstrap complete.
I20260619 12:27:14.220207 22388 ts_tablet_manager.cc:1397] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.004s
I20260619 12:27:14.222954 22388 raft_consensus.cc:348] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 3 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:14.223194 22388 raft_consensus.cc:729] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 3 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92faa47eb6f64d278fd8cf7b03d8f948, State: Initialized, Role: FOLLOWER
I20260619 12:27:14.223702 22388 consensus_queue.cc:260] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 3.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a22c80698d9e40dfba1f2c4799afa8ab" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37247 } } peers { permanent_uuid: "92faa47eb6f64d278fd8cf7b03d8f948" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 35603 } } peers { permanent_uuid: "7092a48aa9f24f279d983e3867e7a10f" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 45135 } }
I20260619 12:27:14.224651 22388 ts_tablet_manager.cc:1428] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948: Time spent starting tablet: real 0.004s user 0.006s sys 0.000s
I20260619 12:27:14.225091 22390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:14.224771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:43323)
0619 12:27:14.224900 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:14.225006 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 12:27:14.225404 22062 tablet_copy_service.cc:342] P 7092a48aa9f24f279d983e3867e7a10f: Request end of tablet copy session 92faa47eb6f64d278fd8cf7b03d8f948-88a517fd16b645aa91f7cf75d7242847 received from {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:44585
I20260619 12:27:14.225548 22062 tablet_copy_service.cc:433] P 7092a48aa9f24f279d983e3867e7a10f: ending tablet copy session 92faa47eb6f64d278fd8cf7b03d8f948-88a517fd16b645aa91f7cf75d7242847 on tablet 88a517fd16b645aa91f7cf75d7242847 with peer 92faa47eb6f64d278fd8cf7b03d8f948
I20260619 12:27:14.225992 22390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:14.225712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:38243)
0619 12:27:14.225823 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:14.225878 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 12:27:14.570948 22396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:14.570549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:60577)
0619 12:27:14.570771 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:14.570834 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":56,"threads_started":1}
I20260619 12:27:14.570967 22310 raft_consensus.cc:1203] T 88a517fd16b645aa91f7cf75d7242847 P 92faa47eb6f64d278fd8cf7b03d8f948 [term 3 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[3.3-3.3] Dedup: 3.3->[]
I20260619 12:27:14.571997 22390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:14.571672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:37037)
0619 12:27:14.571850 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:14.571926 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
I20260619 12:27:14.707741 22397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:14.707348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:50159)
0619 12:27:14.707588 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:14.707646 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":82,"threads_started":1}
I20260619 12:27:15.572793 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:15.572121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:60785)
0619 12:27:15.572593 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:15.572661 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":112,"threads_started":1}
I20260619 12:27:15.573101 22399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:15.572682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:43841)
0619 12:27:15.572954 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:15.573025 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":70,"threads_started":1}
I20260619 12:27:15.573487 22398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:15.573253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:38131)
0619 12:27:15.573343 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:15.573395 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 12:27:15.709139 22400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:15.708709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:32805)
0619 12:27:15.708982 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:15.709043 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":76,"threads_started":1}
I20260619 12:27:16.574620 22402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:16.574165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:58775)
0619 12:27:16.574476 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:16.574533 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":117,"threads_started":1}
I20260619 12:27:16.574611 22401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:16.574165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:39921)
0619 12:27:16.574453 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:16.574513 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":72,"threads_started":1}
I20260619 12:27:16.710438 22403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:16.709887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:45291)
0619 12:27:16.710229 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:16.710310 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":91,"threads_started":1}
I20260619 12:27:17.575884 22405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:17.575460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:56157)
0619 12:27:17.575730 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:17.575789 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":89,"threads_started":1}
I20260619 12:27:17.575927 22404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:17.575460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:58147)
0619 12:27:17.575792 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:17.575844 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":92,"threads_started":1}
I20260619 12:27:17.711798 22406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:17.711349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:37889)
0619 12:27:17.711637 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:17.711701 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":80,"threads_started":1}
I20260619 12:27:18.577150 22407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:18.576694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:51897)
0619 12:27:18.576971 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:18.577047 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":72,"threads_started":1}
I20260619 12:27:18.577181 22408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:18.576767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:40939)
0619 12:27:18.577037 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:18.577090 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":78,"threads_started":1}
I20260619 12:27:18.713163 22409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:18.712704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:42929)
0619 12:27:18.712985 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:18.713050 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":75,"threads_started":1}
I20260619 12:27:19.578503 22410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:19.577995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:54779)
0619 12:27:19.578355 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:19.578413 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":96,"threads_started":1}
I20260619 12:27:19.578660 22411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:19.577995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:60715)
0619 12:27:19.578303 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:19.578402 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":93,"threads_started":1}
I20260619 12:27:19.714684 22412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:19.714097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:55773)
0619 12:27:19.714525 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:19.714587 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":200,"threads_started":1}
I20260619 12:27:20.579770 22413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:20.579301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:41537)
0619 12:27:20.579603 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:20.579666 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":105,"threads_started":1}
I20260619 12:27:20.579962 22414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:20.579516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:39911)
0619 12:27:20.579771 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:20.579850 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":98,"threads_started":1}
I20260619 12:27:20.716116 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:20.715685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:49723)
0619 12:27:20.715934 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:20.715998 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":75,"threads_started":1}
I20260619 12:27:21.581153 22417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:21.580717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:34995)
0619 12:27:21.580979 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:21.581040 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":88,"threads_started":1}
I20260619 12:27:21.581210 22418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:21.580704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:45713)
0619 12:27:21.581051 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:21.581126 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":99,"threads_started":1}
I20260619 12:27:21.717288 22419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:21.716872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:36363)
0619 12:27:21.717131 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:21.717191 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":74,"threads_started":1}
I20260619 12:27:22.582408 22421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:22.581973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:58929)
0619 12:27:22.582258 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:22.582315 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":106,"threads_started":1}
I20260619 12:27:22.582444 22420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:22.581973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:47773)
0619 12:27:22.582301 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:22.582350 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":108,"threads_started":1}
I20260619 12:27:22.718557 22422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:22.718127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:45175)
0619 12:27:22.718395 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:22.718453 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":92,"threads_started":1}
I20260619 12:27:23.583807 22424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:23.583283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:56987)
0619 12:27:23.583661 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:23.583714 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":117,"threads_started":1}
I20260619 12:27:23.583833 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:23.583253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:42109)
0619 12:27:23.583672 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:23.583733 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":124,"threads_started":1}
I20260619 12:27:23.719815 22425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:23.719370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:41059)
0619 12:27:23.719656 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:23.719717 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":83,"threads_started":1}
I20260619 12:27:24.585196 22427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:24.584662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:50067)
0619 12:27:24.585040 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:24.585096 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":306,"thread_start_us":76,"threads_started":1}
I20260619 12:27:24.585214 22426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:24.584646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:55169)
0619 12:27:24.585037 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:24.585107 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":117,"threads_started":1}
I20260619 12:27:24.721124 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:24.720670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.194:36981)
0619 12:27:24.720944 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:24.721008 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":77,"threads_started":1}
I20260619 12:27:25.586701 22429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:25.586157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.195:45589)
0619 12:27:25.586551 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:25.586603 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":100,"threads_started":1}
I20260619 12:27:25.586740 22430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:25.586273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:33275 (local address 127.6.231.193:55903)
0619 12:27:25.586602 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:25.586657 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:33275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":53,"threads_started":1}
I20260619 12:27:25.608309 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 21924
I20260619 12:27:25.614687 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 22091
I20260619 12:27:25.620543 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 22226
2026-06-19T12:27:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19518 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:27:25 dist-test-slave-kr9h krb5kdc[22433](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:27:25 dist-test-slave-kr9h krb5kdc[22433](info): set up 2 sockets
Jun 19 12:27:25 dist-test-slave-kr9h krb5kdc[22433](info): commencing operation
krb5kdc: starting...
W20260619 12:27:26.693753 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:27:26 dist-test-slave-kr9h krb5kdc[22433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:27:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:27:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:27:26.821133 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:34559
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:34559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:27:26.930168 22446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:26.930442 22446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:26.930483 22446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:26.934351 22446 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 12:27:26.934402 22446 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 12:27:26.934418 22446 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 12:27:26.934432 22446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:27:26.934450 22446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:26.934468 22446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:26.934484 22446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:27:26.934499 22446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:27:26.939601 22446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:34559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:34559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:26.940604 22446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:26.941601 22446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:26.950464 22454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:26.950479 22452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:26.950469 22451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:26.950838 22446 server_base.cc:1034] running on GCE node
I20260619 12:27:26.951406 22446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:26.952805 22446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:26.953987 22446 hybrid_clock.cc:648] HybridClock initialized: now 1781872046953970 us; error 32 us; skew 500 ppm
Jun 19 12:27:26 dist-test-slave-kr9h krb5kdc[22433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872046, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:26.956854 22446 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:27:26.957847 22446 webserver.cc:460] Webserver started at http://127.6.231.254:39215/ using document root <none> and password file <none>
I20260619 12:27:26.958405 22446 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:26.958491 22446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:26.958726 22446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:27:26.960805 22446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "b0621e85d9924636b0283821c13a5a65"
format_stamp: "Formatted at 2026-06-19 12:27:26 on dist-test-slave-kr9h"
server_key: "62bd105f9d8a84b24b7587aa60c4b10d"
server_key_iv: "41789141532c2933b9cfc071a7c66218"
server_key_version: "encryptionkey@0"
I20260619 12:27:26.961362 22446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "b0621e85d9924636b0283821c13a5a65"
format_stamp: "Formatted at 2026-06-19 12:27:26 on dist-test-slave-kr9h"
server_key: "62bd105f9d8a84b24b7587aa60c4b10d"
server_key_iv: "41789141532c2933b9cfc071a7c66218"
server_key_version: "encryptionkey@0"
I20260619 12:27:26.965063 22446 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:27:26.967519 22461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:26.968719 22446 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:27:26.968868 22446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "b0621e85d9924636b0283821c13a5a65"
format_stamp: "Formatted at 2026-06-19 12:27:26 on dist-test-slave-kr9h"
server_key: "62bd105f9d8a84b24b7587aa60c4b10d"
server_key_iv: "41789141532c2933b9cfc071a7c66218"
server_key_version: "encryptionkey@0"
I20260619 12:27:26.968981 22446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:26.990121 22446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:26.993307 22446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:26.993538 22446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:27.000195 22446 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:34559
I20260619 12:27:27.000517 22513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:34559 every 8 connection(s)
I20260619 12:27:27.001209 22446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:27:27.004145 22514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:27.007602 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 22446
I20260619 12:27:27.007762 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:27:27.008061 7071 external_mini_cluster.cc:1428] Setting key 48973a75b7a0ae98615fad804aee9b27
I20260619 12:27:27.010274 22514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Bootstrap starting.
I20260619 12:27:27.012980 22514 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:27.013926 22514 log.cc:824] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:27.016045 22514 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: No bootstrap required, opened a new log
Jun 19 12:27:27 dist-test-slave-kr9h krb5kdc[22433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872046, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:27:27.020691 22514 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } }
I20260619 12:27:27.020995 22514 raft_consensus.cc:374] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:27.021054 22514 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0621e85d9924636b0283821c13a5a65, State: Initialized, Role: FOLLOWER
I20260619 12:27:27.021623 22514 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } }
I20260619 12:27:27.021795 22514 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:27:27.021867 22514 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:27:27.021966 22514 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:27.023159 22514 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } }
I20260619 12:27:27.023535 22514 leader_election.cc:302] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0621e85d9924636b0283821c13a5a65; no voters:
I20260619 12:27:27.023865 22514 leader_election.cc:288] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:27:27.023979 22519 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:27:27.024225 22519 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 1 LEADER]: Becoming Leader. State: Replica: b0621e85d9924636b0283821c13a5a65, State: Running, Role: LEADER
I20260619 12:27:27.024296 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.009555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59112 (local address 127.6.231.254:34559)
0619 12:27:27.009982 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:27.010000 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:27.010043 (+ 43us) server_negotiation.cc:407] Connection header received
0619 12:27:27.010870 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:27.010892 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:27.011298 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:27.011678 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:27.012393 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.013971 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.014840 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.015504 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.015662 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.018707 (+ 3045us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:27.018715 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:27.018743 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:27.018777 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:27.020955 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:27.021530 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:27.021535 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:27.021541 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:27.021686 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:27.022006 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:27.022009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:27.022014 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:27.022521 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:27.022739 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:27.022950 (+ 211us) server_negotiation.cc:299] Negotiation successful
0619 12:27:27.023267 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":117,"threads_started":1}
I20260619 12:27:27.024576 22519 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } }
I20260619 12:27:27.025135 22514 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:27:27.025733 22519 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0621e85d9924636b0283821c13a5a65. Latest consensus state: current_term: 1 leader_uuid: "b0621e85d9924636b0283821c13a5a65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } } }
I20260619 12:27:27.025986 22519 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:27.026160 22521 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b0621e85d9924636b0283821c13a5a65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } } }
I20260619 12:27:27.026262 22521 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:27.027220 22527 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:27:27.030027 22527 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:27:27.035303 22527 catalog_manager.cc:1347] Generated new cluster ID: 336742e49aef40a9b6d8c62cc8ffb3b9
I20260619 12:27:27.035388 22527 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:27:27.042665 22527 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:27:27.043702 22527 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:27:27.051954 22527 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Generated new TSK 0
I20260619 12:27:27.052804 22527 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:27:27.089175 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:34559
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:27:27.206327 22540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:27.206627 22540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:27.206681 22540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:27.210892 22540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:27.210968 22540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:27.211031 22540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:27:27.216650 22540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:34559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:27.217763 22540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:27.218773 22540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:27.229182 22548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:27.229174 22546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:27.229351 22540 server_base.cc:1034] running on GCE node
W20260619 12:27:27.229182 22545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:27.230033 22540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:27.230760 22540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:27.231963 22540 hybrid_clock.cc:648] HybridClock initialized: now 1781872047231929 us; error 42 us; skew 500 ppm
Jun 19 12:27:27 dist-test-slave-kr9h krb5kdc[22433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872047, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:27.235976 22540 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:27:27.237272 22540 webserver.cc:460] Webserver started at http://127.6.231.193:43409/ using document root <none> and password file <none>
I20260619 12:27:27.238116 22540 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:27.238245 22540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:27.238524 22540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:27:27.240963 22540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "8539e55d90f8497b8ddf110216e29a72"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "7ec15d280fdc80b421d28820c3e746b5"
server_key_iv: "35e21a8d44e4683550bc2db691614469"
server_key_version: "encryptionkey@0"
I20260619 12:27:27.241621 22540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "8539e55d90f8497b8ddf110216e29a72"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "7ec15d280fdc80b421d28820c3e746b5"
server_key_iv: "35e21a8d44e4683550bc2db691614469"
server_key_version: "encryptionkey@0"
I20260619 12:27:27.245651 22540 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:27:27.248076 22555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:27.249325 22540 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:27:27.249486 22540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "8539e55d90f8497b8ddf110216e29a72"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "7ec15d280fdc80b421d28820c3e746b5"
server_key_iv: "35e21a8d44e4683550bc2db691614469"
server_key_version: "encryptionkey@0"
I20260619 12:27:27.249595 22540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:27.270207 22540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:27.274053 22540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:27.274276 22540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:27.274888 22540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:27.275730 22540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:27:27.275777 22540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:27.275847 22540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:27:27.275878 22540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:27.285071 22540 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:35497
I20260619 12:27:27.285089 22668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:35497 every 8 connection(s)
I20260619 12:27:27.286068 22540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:27:27.286659 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 22540
I20260619 12:27:27.286806 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:27:27.287119 7071 external_mini_cluster.cc:1428] Setting key 54eb770225f6aa9e0bf8a20ae9cd6c9f
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
Jun 19 12:27:27 dist-test-slave-kr9h krb5kdc[22433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872047, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:27:27.301178 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.288104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:34629 (local address 127.6.231.254:34559)
0619 12:27:27.288281 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:27.288285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:27.289662 (+ 1377us) server_negotiation.cc:407] Connection header received
0619 12:27:27.291054 (+ 1392us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:27.291057 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:27.291110 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:27.291191 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:27.292693 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.293368 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.294662 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.294901 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.295018 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.297565 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:27.297569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:27.297572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:27.297606 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:27.299073 (+ 1467us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:27.299604 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:27.299608 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:27.299610 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:27.299715 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:27.300042 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:27.300046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:27.300048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:27.300234 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:27.300351 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:27.300872 (+ 521us) server_negotiation.cc:299] Negotiation successful
0619 12:27:27.301002 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 12:27:27.302055 22671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.288657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:34629)
0619 12:27:27.289499 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:27.289558 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:27.290832 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:27.291367 (+ 535us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:27.291373 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:27.291837 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:27.292529 (+ 692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:27.292551 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.293530 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.293535 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:27.294530 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:27.294540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.295064 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.295068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:27.295253 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.295879 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:27.295895 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:27.297349 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:27.299184 (+ 1835us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:27.299190 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:27.299206 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:27.299496 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:27.299796 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:27.299799 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:27.299801 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:27.299946 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:27.300348 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:27.300354 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:27.300624 (+ 270us) client_negotiation.cc:769] Sending connection context
0619 12:27:27.300821 (+ 197us) client_negotiation.cc:240] Negotiation successful
0619 12:27:27.301155 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":143,"threads_started":1}
I20260619 12:27:27.303342 22669 heartbeater.cc:347] Connected to a master server at 127.6.231.254:34559
I20260619 12:27:27.303570 22669 heartbeater.cc:464] Registering TS with master...
I20260619 12:27:27.304060 22669 heartbeater.cc:511] Master 127.6.231.254:34559 requested a full tablet report, sending...
I20260619 12:27:27.305717 22478 ts_manager.cc:194] Registered new tserver with Master: 8539e55d90f8497b8ddf110216e29a72 (127.6.231.193:35497)
I20260619 12:27:27.307394 22478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:34629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:27:27.316262 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:34559
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:27:27.432394 22674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:27.432689 22674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:27.432734 22674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:27.436857 22674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:27.436929 22674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:27.436990 22674 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:27:27.442376 22674 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:34559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:27.443552 22674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:27.444595 22674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:27.454253 22679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:27.454339 22682 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:27.454298 22680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:27.454473 22674 server_base.cc:1034] running on GCE node
I20260619 12:27:27.454980 22674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:27.455626 22674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:27.456813 22674 hybrid_clock.cc:648] HybridClock initialized: now 1781872047456792 us; error 31 us; skew 500 ppm
Jun 19 12:27:27 dist-test-slave-kr9h krb5kdc[22433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872047, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:27.459872 22674 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:27:27.460819 22674 webserver.cc:460] Webserver started at http://127.6.231.194:46527/ using document root <none> and password file <none>
I20260619 12:27:27.461401 22674 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:27.461549 22674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:27.461789 22674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:27:27.463851 22674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "3bab1a833f7b41b3ac182d27be5b6af7"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "59e24a1957e60834ad62797829fadbfe"
server_key_iv: "43cd99ee3e148d03d26b2eeca58ad4c0"
server_key_version: "encryptionkey@0"
I20260619 12:27:27.464425 22674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "3bab1a833f7b41b3ac182d27be5b6af7"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "59e24a1957e60834ad62797829fadbfe"
server_key_iv: "43cd99ee3e148d03d26b2eeca58ad4c0"
server_key_version: "encryptionkey@0"
I20260619 12:27:27.468390 22674 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260619 12:27:27.471342 22689 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:27.472788 22674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:27:27.472937 22674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "3bab1a833f7b41b3ac182d27be5b6af7"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "59e24a1957e60834ad62797829fadbfe"
server_key_iv: "43cd99ee3e148d03d26b2eeca58ad4c0"
server_key_version: "encryptionkey@0"
I20260619 12:27:27.473062 22674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:27.500180 22674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:27.503875 22674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:27.504093 22674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:27.504724 22674 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:27.505630 22674 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:27:27.505697 22674 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:27.505762 22674 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:27:27.505816 22674 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:27.514381 22674 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:46563
I20260619 12:27:27.514451 22802 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:46563 every 8 connection(s)
I20260619 12:27:27.515323 22674 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:27:27.524608 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 22674
I20260619 12:27:27.524737 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:27:27.525099 7071 external_mini_cluster.cc:1428] Setting key 73c860337dcc221e8748535203d0f1d4
Jun 19 12:27:27 dist-test-slave-kr9h krb5kdc[22433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872047, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:27:27.530799 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.517269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:56357 (local address 127.6.231.254:34559)
0619 12:27:27.517418 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:27.517423 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:27.518608 (+ 1185us) server_negotiation.cc:407] Connection header received
0619 12:27:27.519604 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:27.519608 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:27.519671 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:27.519776 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:27.521057 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.521689 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.522930 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.523162 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.523291 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.526459 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:27.526463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:27.526465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:27.526493 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:27.528078 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:27.528822 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:27.528829 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:27.528833 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:27.528952 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:27.529360 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:27.529366 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:27.529370 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:27.529678 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:27.529806 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:27.530358 (+ 552us) server_negotiation.cc:299] Negotiation successful
0619 12:27:27.530524 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 12:27:27.531539 22805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.517721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:56357)
0619 12:27:27.518432 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:27.518483 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:27.519362 (+ 879us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:27.519966 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:27.519974 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:27.520363 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:27.520891 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:27.520905 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.521836 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.521839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:27.522786 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:27.522795 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.523332 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.523336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:27.523532 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.524165 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:27.524181 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:27.526262 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:27.528204 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:27.528213 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:27.528235 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:27.528647 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:27.529087 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:27.529089 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:27.529091 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:27.529247 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:27.529830 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:27.529835 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:27.530095 (+ 260us) client_negotiation.cc:769] Sending connection context
0619 12:27:27.530264 (+ 169us) client_negotiation.cc:240] Negotiation successful
0619 12:27:27.530560 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":114,"threads_started":1}
I20260619 12:27:27.533022 22803 heartbeater.cc:347] Connected to a master server at 127.6.231.254:34559
I20260619 12:27:27.533305 22803 heartbeater.cc:464] Registering TS with master...
I20260619 12:27:27.533911 22803 heartbeater.cc:511] Master 127.6.231.254:34559 requested a full tablet report, sending...
I20260619 12:27:27.535090 22478 ts_manager.cc:194] Registered new tserver with Master: 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563)
I20260619 12:27:27.535691 22478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:56357
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:27:27.556241 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:34559
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:27:27.666922 22808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:27.667246 22808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:27.667331 22808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:27.671522 22808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:27.671622 22808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:27.671718 22808 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:27:27.677368 22808 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:34559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:27.678547 22808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:27.679639 22808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:27.689589 22813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:27.689616 22814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:27.689589 22816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:27.690021 22808 server_base.cc:1034] running on GCE node
I20260619 12:27:27.690461 22808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:27.691170 22808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:27.692364 22808 hybrid_clock.cc:648] HybridClock initialized: now 1781872047692343 us; error 39 us; skew 500 ppm
Jun 19 12:27:27 dist-test-slave-kr9h krb5kdc[22433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872047, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:27.695230 22808 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:27:27.696188 22808 webserver.cc:460] Webserver started at http://127.6.231.195:44329/ using document root <none> and password file <none>
I20260619 12:27:27.696767 22808 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:27.696866 22808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:27.697099 22808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:27:27.699193 22808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "137c44acca1442a4ad9643b25354c80a"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "a166ab4b4322ecf3789a921d74eb7b13"
server_key_iv: "b950219cddf32b05a9557574c7884be0"
server_key_version: "encryptionkey@0"
I20260619 12:27:27.699759 22808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "137c44acca1442a4ad9643b25354c80a"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "a166ab4b4322ecf3789a921d74eb7b13"
server_key_iv: "b950219cddf32b05a9557574c7884be0"
server_key_version: "encryptionkey@0"
I20260619 12:27:27.703603 22808 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:27:27.706729 22823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:27.708318 22808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:27:27.708429 22808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "137c44acca1442a4ad9643b25354c80a"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "a166ab4b4322ecf3789a921d74eb7b13"
server_key_iv: "b950219cddf32b05a9557574c7884be0"
server_key_version: "encryptionkey@0"
I20260619 12:27:27.708529 22808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:27.729177 22808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:27.732796 22808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:27.732966 22808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:27.733618 22808 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:27.734473 22808 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:27:27.734522 22808 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:27.734563 22808 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:27:27.734578 22808 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:27.744091 22808 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:40207
I20260619 12:27:27.744108 22936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:40207 every 8 connection(s)
I20260619 12:27:27.745051 22808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:27:27.753037 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 22808
I20260619 12:27:27.753186 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:27:27.753605 7071 external_mini_cluster.cc:1428] Setting key 8b4c81616908c6d952b0b8375ec15139
Jun 19 12:27:27 dist-test-slave-kr9h krb5kdc[22433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872047, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:27:27.759486 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.746952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:41869 (local address 127.6.231.254:34559)
0619 12:27:27.747128 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:27.747133 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:27.748376 (+ 1243us) server_negotiation.cc:407] Connection header received
0619 12:27:27.749282 (+ 906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:27.749284 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:27.749338 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:27.749423 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:27.750716 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.751245 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.752497 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.752728 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.752840 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.755756 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:27.755760 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:27.755763 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:27.755797 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:27.757325 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:27.757895 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:27.757901 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:27.757902 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:27.757986 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:27.758362 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:27.758367 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:27.758373 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:27.758554 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:27.758664 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:27.759187 (+ 523us) server_negotiation.cc:299] Negotiation successful
0619 12:27:27.759311 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 12:27:27.760327 22939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.747402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:41869)
0619 12:27:27.748199 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:27.748253 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:27.749088 (+ 835us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:27.749658 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:27.749665 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:27.750038 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:27.750561 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:27.750576 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.751382 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.751385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:27.752314 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:27.752327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.752894 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.752901 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:27.753124 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.754003 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:27.754023 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:27.755583 (+ 1560us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:27.757482 (+ 1899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:27.757490 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:27.757508 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:27.757781 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:27.758081 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:27.758085 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:27.758089 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:27.758250 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:27.758664 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:27.758672 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:27.758943 (+ 271us) client_negotiation.cc:769] Sending connection context
0619 12:27:27.759135 (+ 192us) client_negotiation.cc:240] Negotiation successful
0619 12:27:27.759481 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"thread_start_us":175,"threads_started":1}
I20260619 12:27:27.761574 22937 heartbeater.cc:347] Connected to a master server at 127.6.231.254:34559
I20260619 12:27:27.761945 22937 heartbeater.cc:464] Registering TS with master...
I20260619 12:27:27.762535 22937 heartbeater.cc:511] Master 127.6.231.254:34559 requested a full tablet report, sending...
I20260619 12:27:27.763572 22478 ts_manager.cc:194] Registered new tserver with Master: 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207)
I20260619 12:27:27.764149 22478 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:41869
I20260619 12:27:27.767841 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:27:27.777531 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.769855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59126 (local address 127.6.231.254:34559)
0619 12:27:27.770001 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:27.770006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:27.770178 (+ 172us) server_negotiation.cc:407] Connection header received
0619 12:27:27.770302 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:27.770304 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:27.770344 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:27.770402 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:27.771064 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.771637 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.772542 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.772849 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.772940 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.774394 (+ 1454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:27.774397 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:27.774399 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:27.774422 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:27.775786 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:27.776177 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:27.776180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:27.776181 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:27.776346 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:27.776589 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:27.776591 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:27.776593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:27.776827 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:27.776947 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:27.777167 (+ 220us) server_negotiation.cc:299] Negotiation successful
0619 12:27:27.777322 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:27:27.780328 22478 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59126:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:27:27.783488 22478 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:27:27.798321 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.792315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:40207 (local address 127.0.0.1:35790)
0619 12:27:27.793916 (+ 1601us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:27.793930 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:27.794035 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:27.794443 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:27.794446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:27.794463 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:27.794556 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:27.794562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.795532 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.795535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:27.796765 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:27.796774 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.797703 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.797706 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:27.797834 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.797874 (+ 40us) client_negotiation.cc:769] Sending connection context
0619 12:27:27.797991 (+ 117us) client_negotiation.cc:240] Negotiation successful
0619 12:27:27.798120 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1538,"mutex_wait_us":1,"thread_start_us":91,"threads_started":1}
I20260619 12:27:27.798321 22948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.791996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:46563 (local address 127.0.0.1:60816)
0619 12:27:27.792886 (+ 890us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:27.792942 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:27.793079 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:27.793668 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:27.793674 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:27.793698 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:27.793930 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:27.793942 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.795432 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.795435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:27.796814 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:27.796825 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.797967 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.797972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:27.798066 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.798086 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 12:27:27.798143 (+ 57us) client_negotiation.cc:240] Negotiation successful
0619 12:27:27.798200 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":1,"thread_start_us":88,"threads_started":1}
I20260619 12:27:27.798775 22949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.791996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:35497 (local address 127.0.0.1:47724)
0619 12:27:27.793567 (+ 1571us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:27.793581 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:27.793679 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:27.794230 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:27.794233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:27.794250 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:27.794358 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:27.794364 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.795729 (+ 1365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.795732 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:27.796836 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:27.796842 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.798373 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.798379 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:27.798481 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.798508 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 12:27:27.798578 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 12:27:27.798654 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1512,"mutex_wait_us":202,"thread_start_us":98,"threads_started":1}
I20260619 12:27:27.798784 22951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.792250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60816 (local address 127.6.231.194:46563)
0619 12:27:27.793105 (+ 855us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:27.793114 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:27.793134 (+ 20us) server_negotiation.cc:407] Connection header received
0619 12:27:27.793216 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:27.793221 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:27.793490 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:27.793673 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:27.794074 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.795277 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.796975 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.797862 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.797973 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.798113 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:27.798170 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 12:27:27.798260 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":759,"thread_start_us":176,"threads_started":1}
I20260619 12:27:27.798949 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.792862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35790 (local address 127.6.231.195:40207)
0619 12:27:27.793398 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:27.793407 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:27.794032 (+ 625us) server_negotiation.cc:407] Connection header received
0619 12:27:27.794136 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:27.794142 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:27.794311 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:27.794461 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:27.794650 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.795399 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.796891 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.797579 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.797701 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.797862 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:27.798685 (+ 823us) server_negotiation.cc:299] Negotiation successful
0619 12:27:27.798784 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":135,"threads_started":1}
I20260619 12:27:27.799198 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:27.792691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47724 (local address 127.6.231.193:35497)
0619 12:27:27.793006 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:27.793020 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:27.793782 (+ 762us) server_negotiation.cc:407] Connection header received
0619 12:27:27.793880 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:27.793887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:27.794091 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:27.794242 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:27.794497 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.795601 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.796975 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:27.797854 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:27.797973 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:27.798113 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:27.798950 (+ 837us) server_negotiation.cc:299] Negotiation successful
0619 12:27:27.799062 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":92,"threads_started":1}
I20260619 12:27:27.801717 22737 tablet_service.cc:1467] Processing CreateTablet for tablet bccb9425333f4521b9ec336474f0b5f8 (DEFAULT_TABLE table=test-table [id=d196d09f18a442dbbbec8e173f51bd26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:27.801769 22871 tablet_service.cc:1467] Processing CreateTablet for tablet bccb9425333f4521b9ec336474f0b5f8 (DEFAULT_TABLE table=test-table [id=d196d09f18a442dbbbec8e173f51bd26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:27.801767 22603 tablet_service.cc:1467] Processing CreateTablet for tablet bccb9425333f4521b9ec336474f0b5f8 (DEFAULT_TABLE table=test-table [id=d196d09f18a442dbbbec8e173f51bd26]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:27.802785 22737 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bccb9425333f4521b9ec336474f0b5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:27.802793 22603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bccb9425333f4521b9ec336474f0b5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:27.802807 22871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bccb9425333f4521b9ec336474f0b5f8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:27.809233 22954 tablet_bootstrap.cc:492] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7: Bootstrap starting.
I20260619 12:27:27.810717 22956 tablet_bootstrap.cc:492] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a: Bootstrap starting.
I20260619 12:27:27.812264 22954 tablet_bootstrap.cc:654] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:27.813212 22954 log.cc:824] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:27.813580 22956 tablet_bootstrap.cc:654] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:27.813832 22955 tablet_bootstrap.cc:492] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Bootstrap starting.
I20260619 12:27:27.814435 22956 log.cc:824] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:27.815806 22954 tablet_bootstrap.cc:492] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7: No bootstrap required, opened a new log
I20260619 12:27:27.816005 22955 tablet_bootstrap.cc:654] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:27.816006 22954 ts_tablet_manager.cc:1397] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260619 12:27:27.816437 22956 tablet_bootstrap.cc:492] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a: No bootstrap required, opened a new log
I20260619 12:27:27.816623 22956 ts_tablet_manager.cc:1397] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 12:27:27.816789 22955 log.cc:824] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:27.818406 22955 tablet_bootstrap.cc:492] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: No bootstrap required, opened a new log
I20260619 12:27:27.818611 22955 ts_tablet_manager.cc:1397] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 12:27:27.820120 22954 raft_consensus.cc:348] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } }
I20260619 12:27:27.820449 22954 raft_consensus.cc:374] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:27.820503 22954 raft_consensus.cc:729] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bab1a833f7b41b3ac182d27be5b6af7, State: Initialized, Role: FOLLOWER
I20260619 12:27:27.820391 22956 raft_consensus.cc:348] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } }
I20260619 12:27:27.820645 22956 raft_consensus.cc:374] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:27.820698 22956 raft_consensus.cc:729] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 137c44acca1442a4ad9643b25354c80a, State: Initialized, Role: FOLLOWER
I20260619 12:27:27.820987 22954 consensus_queue.cc:260] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } }
I20260619 12:27:27.821209 22956 consensus_queue.cc:260] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } }
I20260619 12:27:27.822070 22956 ts_tablet_manager.cc:1428] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 12:27:27.822070 22954 ts_tablet_manager.cc:1428] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260619 12:27:27.822468 22937 heartbeater.cc:503] Master 127.6.231.254:34559 was elected leader, sending a full tablet report...
I20260619 12:27:27.822538 22803 heartbeater.cc:503] Master 127.6.231.254:34559 was elected leader, sending a full tablet report...
I20260619 12:27:27.822542 22955 raft_consensus.cc:348] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } }
I20260619 12:27:27.822801 22955 raft_consensus.cc:374] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:27.822853 22955 raft_consensus.cc:729] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8539e55d90f8497b8ddf110216e29a72, State: Initialized, Role: FOLLOWER
I20260619 12:27:27.823247 22955 consensus_queue.cc:260] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } }
I20260619 12:27:27.823962 22669 heartbeater.cc:503] Master 127.6.231.254:34559 was elected leader, sending a full tablet report...
I20260619 12:27:27.824222 22955 ts_tablet_manager.cc:1428] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
W20260619 12:27:27.996045 22938 tablet.cc:2369] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:27:28.016505 22804 tablet.cc:2369] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:27:28.037519 22670 tablet.cc:2369] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:27:28.060091 22961 raft_consensus.cc:482] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:27:28.060469 22961 raft_consensus.cc:504] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } }
I20260619 12:27:28.061774 22961 leader_election.cc:288] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563), 8539e55d90f8497b8ddf110216e29a72 (127.6.231.193:35497)
I20260619 12:27:28.065410 22951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:28.062115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:56981 (local address 127.6.231.194:46563)
0619 12:27:28.062276 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:28.062281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:28.062356 (+ 75us) server_negotiation.cc:407] Connection header received
0619 12:27:28.062531 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:28.062533 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:28.062577 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:28.062663 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:28.062925 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:28.063421 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:28.064468 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:28.064850 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:28.064985 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:28.065038 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:28.065220 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 12:27:28.065287 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 12:27:28.065410 22939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:28.062035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:46563 (local address 127.6.231.195:56981)
0619 12:27:28.062269 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:28.062284 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:28.062400 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:28.062670 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:28.062673 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:28.062689 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:28.062805 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:28.062810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:28.063528 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:28.063534 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:28.064293 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:28.064300 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:28.064973 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:28.064976 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:28.065081 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:28.065108 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 12:27:28.065194 (+ 86us) client_negotiation.cc:240] Negotiation successful
0619 12:27:28.065269 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260619 12:27:28.065579 22952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:28.062202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:49769 (local address 127.6.231.193:35497)
0619 12:27:28.062336 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:28.062341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:28.062555 (+ 214us) server_negotiation.cc:407] Connection header received
0619 12:27:28.062723 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:28.062726 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:28.062771 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:28.062856 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:28.063096 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:28.063602 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:28.064608 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:28.065006 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:28.065132 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:28.065180 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:28.065356 (+ 176us) server_negotiation.cc:299] Negotiation successful
0619 12:27:28.065426 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 12:27:28.065610 22963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:28.062074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:35497 (local address 127.6.231.195:49769)
0619 12:27:28.062460 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:28.062471 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:28.062594 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:28.062864 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:28.062867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:28.062881 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:28.062993 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:28.062998 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:28.063726 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:28.063731 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:28.064488 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:28.064496 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:28.065124 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:28.065127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:28.065225 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:28.065246 (+ 21us) client_negotiation.cc:769] Sending connection context
0619 12:27:28.065342 (+ 96us) client_negotiation.cc:240] Negotiation successful
0619 12:27:28.065409 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":138,"threads_started":1}
I20260619 12:27:28.066154 22757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bccb9425333f4521b9ec336474f0b5f8" candidate_uuid: "137c44acca1442a4ad9643b25354c80a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" is_pre_election: true
I20260619 12:27:28.066219 22623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bccb9425333f4521b9ec336474f0b5f8" candidate_uuid: "137c44acca1442a4ad9643b25354c80a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8539e55d90f8497b8ddf110216e29a72" is_pre_election: true
I20260619 12:27:28.066495 22623 raft_consensus.cc:2454] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 137c44acca1442a4ad9643b25354c80a in term 0.
I20260619 12:27:28.066495 22757 raft_consensus.cc:2454] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 137c44acca1442a4ad9643b25354c80a in term 0.
I20260619 12:27:28.067003 22826 leader_election.cc:302] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 137c44acca1442a4ad9643b25354c80a, 3bab1a833f7b41b3ac182d27be5b6af7; no voters:
I20260619 12:27:28.067220 22961 raft_consensus.cc:2781] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:27:28.067319 22961 raft_consensus.cc:482] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:27:28.067378 22961 raft_consensus.cc:3037] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:28.068481 22961 raft_consensus.cc:504] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } }
I20260619 12:27:28.068883 22961 leader_election.cc:288] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [CANDIDATE]: Term 1 election: Requested vote from peers 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563), 8539e55d90f8497b8ddf110216e29a72 (127.6.231.193:35497)
I20260619 12:27:28.069232 22757 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bccb9425333f4521b9ec336474f0b5f8" candidate_uuid: "137c44acca1442a4ad9643b25354c80a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bab1a833f7b41b3ac182d27be5b6af7"
I20260619 12:27:28.069252 22623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bccb9425333f4521b9ec336474f0b5f8" candidate_uuid: "137c44acca1442a4ad9643b25354c80a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8539e55d90f8497b8ddf110216e29a72"
I20260619 12:27:28.069366 22757 raft_consensus.cc:3037] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:28.069404 22623 raft_consensus.cc:3037] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:28.070408 22623 raft_consensus.cc:2454] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 137c44acca1442a4ad9643b25354c80a in term 1.
I20260619 12:27:28.070420 22757 raft_consensus.cc:2454] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 137c44acca1442a4ad9643b25354c80a in term 1.
I20260619 12:27:28.070747 22824 leader_election.cc:302] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 137c44acca1442a4ad9643b25354c80a, 8539e55d90f8497b8ddf110216e29a72; no voters:
I20260619 12:27:28.070940 22961 raft_consensus.cc:2781] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:27:28.071163 22961 raft_consensus.cc:686] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [term 1 LEADER]: Becoming Leader. State: Replica: 137c44acca1442a4ad9643b25354c80a, State: Running, Role: LEADER
I20260619 12:27:28.071512 22961 consensus_queue.cc:237] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } }
I20260619 12:27:28.074749 22477 catalog_manager.cc:5495] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a reported cstate change: term changed from 0 to 1, leader changed from <none> to 137c44acca1442a4ad9643b25354c80a (127.6.231.195). New cstate: current_term: 1 leader_uuid: "137c44acca1442a4ad9643b25354c80a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } health_report { overall_health: HEALTHY } } }
I20260619 12:27:28.091784 22953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:28.088311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35800 (local address 127.6.231.195:40207)
0619 12:27:28.088461 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:28.088465 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:28.088478 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:27:28.088564 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:28.088566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:28.088617 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:28.088701 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:27:28.088936 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:28.089549 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:28.090428 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:28.090637 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:28.090721 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:28.091016 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:27:28.091458 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:27:28.091550 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:28.091607 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 12:27:28.091657 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 12:27:28.097664 22757 raft_consensus.cc:1261] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 1 FOLLOWER]: Refusing update from remote peer 137c44acca1442a4ad9643b25354c80a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:27:28.097664 22623 raft_consensus.cc:1261] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Refusing update from remote peer 137c44acca1442a4ad9643b25354c80a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:27:28.098340 22961 consensus_queue.cc:1035] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:27:28.098532 22964 consensus_queue.cc:1035] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:27:28.107048 22968 mvcc.cc:205] Tried to move back new op lower bound from 7298547909000544256 to 7298547908903313408. Current Snapshot: MvccSnapshot[applied={T|T < 7298547909000544256}]
I20260619 12:27:28.115770 22476 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:27:28.115964 22476 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59126:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:27:28.118649 22476 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Sending DeleteTablet for 3 replicas of tablet bccb9425333f4521b9ec336474f0b5f8
I20260619 12:27:28.119541 22737 tablet_service.cc:1514] Processing DeleteTablet for tablet bccb9425333f4521b9ec336474f0b5f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:28 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:60816
I20260619 12:27:28.119673 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 22540
I20260619 12:27:28.119659 22603 tablet_service.cc:1514] Processing DeleteTablet for tablet bccb9425333f4521b9ec336474f0b5f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:28 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:47724
I20260619 12:27:28.121861 22977 tablet_replica.cc:330] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7: stopping tablet replica
I20260619 12:27:28.121986 22871 tablet_service.cc:1514] Processing DeleteTablet for tablet bccb9425333f4521b9ec336474f0b5f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:28 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:35790
I20260619 12:27:28.122234 22977 raft_consensus.cc:2229] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:27:28.122638 22977 raft_consensus.cc:2258] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:27:28.122736 22978 tablet_replica.cc:330] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a: stopping tablet replica
I20260619 12:27:28.123050 22978 raft_consensus.cc:2229] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:27:28.123469 22978 raft_consensus.cc:2258] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:27:28.124162 22977 ts_tablet_manager.cc:1905] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:27:28.124939 22978 ts_tablet_manager.cc:1905] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:27:28.127075 22977 ts_tablet_manager.cc:1918] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:27:28.127182 22977 log.cc:1196] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/bccb9425333f4521b9ec336474f0b5f8
I20260619 12:27:28.127578 22977 ts_tablet_manager.cc:1939] T bccb9425333f4521b9ec336474f0b5f8 P 3bab1a833f7b41b3ac182d27be5b6af7: Deleting consensus metadata
I20260619 12:27:28.128228 22978 ts_tablet_manager.cc:1918] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:27:28.128319 22978 log.cc:1196] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/bccb9425333f4521b9ec336474f0b5f8
W20260619 12:27:28.128901 22462 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:35497 (error 108)
I20260619 12:27:28.128971 22978 ts_tablet_manager.cc:1939] T bccb9425333f4521b9ec336474f0b5f8 P 137c44acca1442a4ad9643b25354c80a: Deleting consensus metadata
I20260619 12:27:28.129056 22464 catalog_manager.cc:4841] TS 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563): tablet bccb9425333f4521b9ec336474f0b5f8 (table test-table [id=d196d09f18a442dbbbec8e173f51bd26]) successfully deleted
I20260619 12:27:28.129920 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 22674
I20260619 12:27:28.130761 22465 catalog_manager.cc:4841] TS 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207): tablet bccb9425333f4521b9ec336474f0b5f8 (table test-table [id=d196d09f18a442dbbbec8e173f51bd26]) successfully deleted
I20260619 12:27:28.134361 22950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:28.134048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:35497 (local address 127.0.0.1:47740)
0619 12:27:28.134194 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:28.134271 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:35497: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260619 12:27:28.134603 22462 catalog_manager.cc:4584] TS 8539e55d90f8497b8ddf110216e29a72 (127.6.231.193:35497): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bccb9425333f4521b9ec336474f0b5f8: Network error: Client connection negotiation failed: client connection to 127.6.231.193:35497: connect: Connection refused (error 111)
I20260619 12:27:28.138820 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 22808
I20260619 12:27:28.144678 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 22446
I20260619 12:27:48.152034 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:34559
--webserver_interface=127.6.231.254
--webserver_port=39215
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:34559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:27:48.265837 22980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:48.266201 22980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:48.266279 22980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:48.270632 22980 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 12:27:48.270731 22980 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 12:27:48.270761 22980 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 12:27:48.270793 22980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:27:48.270830 22980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:48.270866 22980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:48.270902 22980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:27:48.270941 22980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:27:48.276613 22980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:34559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:34559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=39215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:48.277822 22980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:48.278951 22980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:48.293327 22987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:48.293382 22986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:48.293547 22980 server_base.cc:1034] running on GCE node
W20260619 12:27:48.293372 22989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:48.294355 22980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:48.295747 22980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:48.296974 22980 hybrid_clock.cc:648] HybridClock initialized: now 1781872068296929 us; error 65 us; skew 500 ppm
Jun 19 12:27:48 dist-test-slave-kr9h krb5kdc[22433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872068, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:48.300663 22980 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:27:48.301856 22980 webserver.cc:460] Webserver started at http://127.6.231.254:39215/ using document root <none> and password file <none>
I20260619 12:27:48.302511 22980 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:48.302630 22980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:48.307060 22980 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:27:48.309615 22996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:48.311023 22980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 12:27:48.311196 22980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "b0621e85d9924636b0283821c13a5a65"
format_stamp: "Formatted at 2026-06-19 12:27:26 on dist-test-slave-kr9h"
server_key: "62bd105f9d8a84b24b7587aa60c4b10d"
server_key_iv: "41789141532c2933b9cfc071a7c66218"
server_key_version: "encryptionkey@0"
I20260619 12:27:48.311862 22980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:48.337060 22980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:48.340644 22980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:48.340879 22980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:48.348327 22980 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:34559
I20260619 12:27:48.348657 23048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:34559 every 8 connection(s)
I20260619 12:27:48.349526 22980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:27:48.354044 23049 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:27:48.355873 23049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Bootstrap starting.
I20260619 12:27:48.359506 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 22980
I20260619 12:27:48.360042 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:35497
--local_ip_for_outbound_sockets=127.6.231.193
--tserver_master_addrs=127.6.231.254:34559
--webserver_port=43409
--webserver_interface=127.6.231.193
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:27:48.374437 23049 log.cc:824] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:48.383848 23049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:27:48.384313 23049 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Bootstrap complete.
I20260619 12:27:48.388509 23049 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } }
I20260619 12:27:48.388926 23049 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0621e85d9924636b0283821c13a5a65, State: Initialized, Role: FOLLOWER
I20260619 12:27:48.389547 23049 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } }
I20260619 12:27:48.389701 23049 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:27:48.389787 23049 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:27:48.389910 23049 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 1 FOLLOWER]: Advancing to term 2
I20260619 12:27:48.391917 23049 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } }
I20260619 12:27:48.392302 23049 leader_election.cc:302] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0621e85d9924636b0283821c13a5a65; no voters:
I20260619 12:27:48.392683 23049 leader_election.cc:288] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260619 12:27:48.392796 23053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 2 FOLLOWER]: Leader election won for term 2
I20260619 12:27:48.393018 23053 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 2 LEADER]: Becoming Leader. State: Replica: b0621e85d9924636b0283821c13a5a65, State: Running, Role: LEADER
I20260619 12:27:48.393312 23053 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } }
I20260619 12:27:48.394023 23049 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:27:48.394701 23054 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "b0621e85d9924636b0283821c13a5a65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } } }
I20260619 12:27:48.394829 23054 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:48.395582 23055 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0621e85d9924636b0283821c13a5a65. Latest consensus state: current_term: 2 leader_uuid: "b0621e85d9924636b0283821c13a5a65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } } }
I20260619 12:27:48.395692 23055 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: This master's current role is: LEADER
I20260619 12:27:48.395941 23060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:27:48.400966 23060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:27:48.401643 23060 catalog_manager.cc:1259] Loaded cluster ID: 336742e49aef40a9b6d8c62cc8ffb3b9
I20260619 12:27:48.401707 23060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:27:48.404131 23060 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:27:48.404809 23060 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Loaded TSK: 0
I20260619 12:27:48.419952 23060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Generated new TSK 1
I20260619 12:27:48.420243 23060 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:27:48.484751 23051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:48.485011 23051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:48.485075 23051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:48.489377 23051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:48.489514 23051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:48.489660 23051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:27:48.495378 23051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:35497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=43409
--tserver_master_addrs=127.6.231.254:34559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:48.496573 23051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:48.497721 23051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:48.507786 23079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:48.507768 23076 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:48.507763 23077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:48.508387 23051 server_base.cc:1034] running on GCE node
I20260619 12:27:48.508864 23051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:48.509671 23051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:48.510888 23051 hybrid_clock.cc:648] HybridClock initialized: now 1781872068510870 us; error 34 us; skew 500 ppm
Jun 19 12:27:48 dist-test-slave-kr9h krb5kdc[22433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872068, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:48.514213 23051 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:27:48.515313 23051 webserver.cc:460] Webserver started at http://127.6.231.193:43409/ using document root <none> and password file <none>
I20260619 12:27:48.516134 23051 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:48.516254 23051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:48.520485 23051 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:27:48.523053 23086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:48.524364 23051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 12:27:48.524528 23051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "8539e55d90f8497b8ddf110216e29a72"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "7ec15d280fdc80b421d28820c3e746b5"
server_key_iv: "35e21a8d44e4683550bc2db691614469"
server_key_version: "encryptionkey@0"
I20260619 12:27:48.525121 23051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:48.550877 23051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:48.554333 23051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:48.554591 23051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:48.555245 23051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:48.556443 23093 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:27:48.560696 23051 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:27:48.560809 23051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260619 12:27:48.560896 23051 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:27:48.562742 23051 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:27:48.562827 23051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260619 12:27:48.563169 23093 tablet_bootstrap.cc:492] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Bootstrap starting.
I20260619 12:27:48.573949 23051 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:35497
I20260619 12:27:48.574029 23200 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:35497 every 8 connection(s)
I20260619 12:27:48.575003 23051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:27:48.578094 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 23051
I20260619 12:27:48.578748 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:46563
--local_ip_for_outbound_sockets=127.6.231.194
--tserver_master_addrs=127.6.231.254:34559
--webserver_port=46527
--webserver_interface=127.6.231.194
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:27:48.582477 23093 log.cc:824] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Log is configured to *not* fsync() on all Append() calls
Jun 19 12:27:48 dist-test-slave-kr9h krb5kdc[22433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872068, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:27:48.590914 23093 tablet_bootstrap.cc:492] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:27:48.591384 23093 tablet_bootstrap.cc:492] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Bootstrap complete.
I20260619 12:27:48.592725 23093 ts_tablet_manager.cc:1397] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Time spent bootstrapping tablet: real 0.030s user 0.026s sys 0.000s
I20260619 12:27:48.595188 23204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:48.577126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:50551)
0619 12:27:48.577845 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:48.577900 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:48.578815 (+ 915us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:48.580128 (+ 1313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:48.580136 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:48.580663 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:48.581377 (+ 714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:48.581401 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:48.583254 (+ 1853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:48.583257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:48.584543 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:48.584553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:48.585739 (+ 1186us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:48.585743 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:48.585930 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:48.586712 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:48.586732 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:48.588800 (+ 2068us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:48.591562 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:48.591569 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:48.591584 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:48.591923 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:48.592377 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:48.592383 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:48.592389 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:48.592561 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:48.593342 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:48.593349 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:48.593710 (+ 361us) client_negotiation.cc:769] Sending connection context
0619 12:27:48.593909 (+ 199us) client_negotiation.cc:240] Negotiation successful
0619 12:27:48.594242 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":140,"threads_started":1}
I20260619 12:27:48.595299 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:48.577235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:50551 (local address 127.6.231.254:34559)
0619 12:27:48.577679 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:48.577700 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:48.578117 (+ 417us) server_negotiation.cc:407] Connection header received
0619 12:27:48.579401 (+ 1284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:48.579411 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:48.579717 (+ 306us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:48.580088 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:48.581623 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:48.583052 (+ 1429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:48.584693 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:48.585576 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:48.585782 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:48.589058 (+ 3276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:48.589065 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:48.589082 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:48.589115 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:48.591427 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:48.592060 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:48.592069 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:48.592077 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:48.592233 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:48.592675 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:48.592678 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:48.592680 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:48.593214 (+ 534us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:48.593346 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:48.594054 (+ 708us) server_negotiation.cc:299] Negotiation successful
0619 12:27:48.594360 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":125,"threads_started":1}
I20260619 12:27:48.595885 23093 raft_consensus.cc:348] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } }
I20260619 12:27:48.596345 23093 raft_consensus.cc:729] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8539e55d90f8497b8ddf110216e29a72, State: Initialized, Role: FOLLOWER
I20260619 12:27:48.596855 23093 consensus_queue.cc:260] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } }
I20260619 12:27:48.597927 23093 ts_tablet_manager.cc:1428] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
I20260619 12:27:48.598567 23201 heartbeater.cc:347] Connected to a master server at 127.6.231.254:34559
I20260619 12:27:48.598821 23201 heartbeater.cc:464] Registering TS with master...
I20260619 12:27:48.599355 23201 heartbeater.cc:511] Master 127.6.231.254:34559 requested a full tablet report, sending...
I20260619 12:27:48.602113 23013 ts_manager.cc:194] Registered new tserver with Master: 8539e55d90f8497b8ddf110216e29a72 (127.6.231.193:35497)
I20260619 12:27:48.605901 23013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:50551
I20260619 12:27:48.608765 23201 heartbeater.cc:503] Master 127.6.231.254:34559 was elected leader, sending a full tablet report...
Jun 19 12:27:48 dist-test-slave-kr9h krb5kdc[22433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872068, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.193@KRBTEST.COM
I20260619 12:27:48.617542 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:48.605133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:35497 (local address 127.0.0.1:38488)
0619 12:27:48.605932 (+ 799us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:48.605979 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:48.606187 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:48.606892 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:48.606898 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:48.607322 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:48.607454 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:48.607461 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:48.608884 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:48.608887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:48.609856 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:48.609868 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:48.610735 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:48.610738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:48.610845 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:48.611455 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:48.611470 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:48.613110 (+ 1640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:48.615407 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:48.615416 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:48.615421 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:48.615748 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:48.616117 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:48.616120 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:48.616122 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:48.616217 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:48.616837 (+ 620us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:48.616842 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:48.616964 (+ 122us) client_negotiation.cc:769] Sending connection context
0619 12:27:48.617099 (+ 135us) client_negotiation.cc:240] Negotiation successful
0619 12:27:48.617271 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":629,"thread_start_us":134,"threads_started":1}
I20260619 12:27:48.617583 23209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:48.605231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38488 (local address 127.6.231.193:35497)
0619 12:27:48.605665 (+ 434us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:48.605677 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:48.606067 (+ 390us) server_negotiation.cc:407] Connection header received
0619 12:27:48.606376 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:48.606382 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:48.606617 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:48.606791 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:48.607601 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:48.608741 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:48.610022 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:48.610611 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:48.610705 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:48.613273 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:48.613289 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:48.613298 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:48.613326 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:48.615260 (+ 1934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:48.615877 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:48.615884 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:48.615886 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:48.615991 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:48.616315 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:48.616319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:48.616321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:48.616685 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:48.616830 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:48.617210 (+ 380us) server_negotiation.cc:299] Negotiation successful
0619 12:27:48.617379 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":114,"threads_started":1}
I20260619 12:27:48.618764 23135 tablet_service.cc:1514] Processing DeleteTablet for tablet bccb9425333f4521b9ec336474f0b5f8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:28 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:38488
I20260619 12:27:48.619724 23210 tablet_replica.cc:330] stopping tablet replica
I20260619 12:27:48.620075 23210 raft_consensus.cc:2229] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:27:48.620347 23210 raft_consensus.cc:2258] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:27:48.621537 23210 ts_tablet_manager.cc:1905] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:27:48.626515 23210 ts_tablet_manager.cc:1918] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:27:48.626673 23210 log.cc:1196] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal/wals/bccb9425333f4521b9ec336474f0b5f8
I20260619 12:27:48.626952 23210 ts_tablet_manager.cc:1939] T bccb9425333f4521b9ec336474f0b5f8 P 8539e55d90f8497b8ddf110216e29a72: Deleting consensus metadata
I20260619 12:27:48.629339 22997 catalog_manager.cc:4841] TS 8539e55d90f8497b8ddf110216e29a72 (127.6.231.193:35497): tablet bccb9425333f4521b9ec336474f0b5f8 (table test-table [id=d196d09f18a442dbbbec8e173f51bd26]) successfully deleted
W20260619 12:27:48.705201 23205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:48.705482 23205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:48.705605 23205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:48.709664 23205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:48.709731 23205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:48.709792 23205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:27:48.715257 23205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:46563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=46527
--tserver_master_addrs=127.6.231.254:34559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:48.716545 23205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:48.717574 23205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:48.727877 23216 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:48.727877 23215 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:48.727911 23218 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:48.727954 23205 server_base.cc:1034] running on GCE node
I20260619 12:27:48.728555 23205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:48.729372 23205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:48.730590 23205 hybrid_clock.cc:648] HybridClock initialized: now 1781872068730576 us; error 28 us; skew 500 ppm
Jun 19 12:27:48 dist-test-slave-kr9h krb5kdc[22433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872068, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:48.738144 23205 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:27:48.739120 23205 webserver.cc:460] Webserver started at http://127.6.231.194:46527/ using document root <none> and password file <none>
I20260619 12:27:48.739676 23205 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:48.739750 23205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:48.743754 23205 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:27:48.746094 23225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:48.747453 23205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:27:48.747576 23205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "3bab1a833f7b41b3ac182d27be5b6af7"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "59e24a1957e60834ad62797829fadbfe"
server_key_iv: "43cd99ee3e148d03d26b2eeca58ad4c0"
server_key_version: "encryptionkey@0"
I20260619 12:27:48.748163 23205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:48.790920 23205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:48.794647 23205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:48.794872 23205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:48.795609 23205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:48.796557 23205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:27:48.796635 23205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:48.796712 23205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:27:48.796764 23205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:48.806382 23205 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:46563
I20260619 12:27:48.806398 23338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:46563 every 8 connection(s)
I20260619 12:27:48.807461 23205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:27:48.817265 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 23205
Jun 19 12:27:48 dist-test-slave-kr9h krb5kdc[22433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872068, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:27:48.817780 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:40207
--local_ip_for_outbound_sockets=127.6.231.195
--tserver_master_addrs=127.6.231.254:34559
--webserver_port=44329
--webserver_interface=127.6.231.195
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:27:48.822475 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:48.809382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:34911 (local address 127.6.231.254:34559)
0619 12:27:48.809571 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:48.809576 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:48.810665 (+ 1089us) server_negotiation.cc:407] Connection header received
0619 12:27:48.811486 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:48.811488 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:48.811539 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:48.811633 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:48.812885 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:48.813470 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:48.814725 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:48.814964 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:48.815076 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:48.818055 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:48.818060 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:48.818063 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:48.818095 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:48.819975 (+ 1880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:48.820520 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:48.820526 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:48.820531 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:48.820627 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:48.820976 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:48.820982 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:48.820986 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:48.821265 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:48.821371 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:48.822058 (+ 687us) server_negotiation.cc:299] Negotiation successful
0619 12:27:48.822224 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 12:27:48.823285 23341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:48.809822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:34911)
0619 12:27:48.810492 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:48.810535 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:48.811302 (+ 767us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:48.811819 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:48.811826 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:48.812183 (+ 357us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:48.812701 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:48.812715 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:48.813649 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:48.813652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:48.814572 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:48.814581 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:48.815128 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:48.815131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:48.815306 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:48.816107 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:48.816139 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:48.817824 (+ 1685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:48.820129 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:48.820134 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:48.820148 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:48.820410 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:48.820732 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:48.820734 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:48.820735 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:48.820873 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:48.821401 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:48.821407 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:48.821764 (+ 357us) client_negotiation.cc:769] Sending connection context
0619 12:27:48.821986 (+ 222us) client_negotiation.cc:240] Negotiation successful
0619 12:27:48.822324 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":112,"threads_started":1}
I20260619 12:27:48.824695 23339 heartbeater.cc:347] Connected to a master server at 127.6.231.254:34559
I20260619 12:27:48.824986 23339 heartbeater.cc:464] Registering TS with master...
I20260619 12:27:48.825591 23339 heartbeater.cc:511] Master 127.6.231.254:34559 requested a full tablet report, sending...
I20260619 12:27:48.826730 23013 ts_manager.cc:194] Registered new tserver with Master: 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563)
I20260619 12:27:48.827340 23013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:34911
W20260619 12:27:48.935148 23342 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:27:48.935436 23342 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:27:48.935480 23342 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:27:48.939807 23342 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:27:48.939878 23342 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:27:48.939980 23342 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:27:48.945614 23342 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:40207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=44329
--tserver_master_addrs=127.6.231.254:34559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23342
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:27:48.946733 23342 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:27:48.947803 23342 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:27:48.958191 23347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:48.958204 23350 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:27:48.958245 23348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:27:48.958519 23342 server_base.cc:1034] running on GCE node
I20260619 12:27:48.959010 23342 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:27:48.959689 23342 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:27:48.960892 23342 hybrid_clock.cc:648] HybridClock initialized: now 1781872068960877 us; error 26 us; skew 500 ppm
Jun 19 12:27:48 dist-test-slave-kr9h krb5kdc[22433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872068, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:27:48.963984 23342 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:27:48.964994 23342 webserver.cc:460] Webserver started at http://127.6.231.195:44329/ using document root <none> and password file <none>
I20260619 12:27:48.965631 23342 fs_manager.cc:359] Metadata directory not provided
I20260619 12:27:48.965798 23342 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:27:48.969848 23342 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260619 12:27:48.972048 23357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:48.973500 23342 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:27:48.973639 23342 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "137c44acca1442a4ad9643b25354c80a"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "a166ab4b4322ecf3789a921d74eb7b13"
server_key_iv: "b950219cddf32b05a9557574c7884be0"
server_key_version: "encryptionkey@0"
I20260619 12:27:48.974255 23342 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:27:48.987063 23342 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:27:48.990494 23342 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:27:48.990722 23342 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:27:48.991335 23342 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:27:48.992187 23342 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:27:48.992266 23342 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:48.992332 23342 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:27:48.992386 23342 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:27:49.002000 23342 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:40207
I20260619 12:27:49.002024 23470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:40207 every 8 connection(s)
I20260619 12:27:49.003041 23342 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:27:49.005332 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 23342
Jun 19 12:27:49 dist-test-slave-kr9h krb5kdc[22433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872068, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:27:49.015324 23474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.006145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44028 (local address 127.6.231.254:34559)
0619 12:27:49.006363 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:49.006367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:49.006716 (+ 349us) server_negotiation.cc:407] Connection header received
0619 12:27:49.006849 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:49.006852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:49.006902 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:49.006997 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:49.007811 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.008601 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.009622 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.009896 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.010032 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.011544 (+ 1512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:49.011547 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:49.011549 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:49.011571 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:49.013366 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:49.013912 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:49.013914 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:49.013915 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:49.013992 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:49.014380 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:49.014385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:49.014386 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:49.014581 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:49.014762 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:49.015001 (+ 239us) server_negotiation.cc:299] Negotiation successful
0619 12:27:49.015132 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":73,"threads_started":1}
I20260619 12:27:49.019465 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.004982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:38803 (local address 127.6.231.254:34559)
0619 12:27:49.005157 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:49.005160 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:49.006612 (+ 1452us) server_negotiation.cc:407] Connection header received
0619 12:27:49.007654 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:49.007660 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:49.007714 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:49.007796 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:49.009215 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.009875 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.011280 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.011617 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.011696 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.015455 (+ 3759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:49.015458 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:49.015461 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:49.015484 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:49.017261 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:49.017972 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:49.017975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:49.017976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:49.018060 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:49.018429 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:49.018431 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:49.018432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:49.018684 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:49.018769 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:49.019259 (+ 490us) server_negotiation.cc:299] Negotiation successful
0619 12:27:49.019357 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260619 12:27:49.020364 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.005558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:38803)
0619 12:27:49.006438 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:49.006482 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:49.007439 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:49.008009 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:49.008019 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:49.008516 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:27:49.009063 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:49.009076 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.010004 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.010007 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:49.011126 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:49.011143 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.011772 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.011778 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:49.011988 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.012634 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:27:49.012660 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:27:49.015274 (+ 2614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:27:49.017416 (+ 2142us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:49.017462 (+ 46us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:49.017481 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:49.017846 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:49.018170 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:27:49.018173 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:27:49.018174 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:27:49.018324 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:27:49.018788 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:27:49.018792 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:27:49.019034 (+ 242us) client_negotiation.cc:769] Sending connection context
0619 12:27:49.019205 (+ 171us) client_negotiation.cc:240] Negotiation successful
0619 12:27:49.019462 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":141,"threads_started":1}
I20260619 12:27:49.021786 23471 heartbeater.cc:347] Connected to a master server at 127.6.231.254:34559
I20260619 12:27:49.022034 23471 heartbeater.cc:464] Registering TS with master...
I20260619 12:27:49.022535 23471 heartbeater.cc:511] Master 127.6.231.254:34559 requested a full tablet report, sending...
I20260619 12:27:49.023701 23013 ts_manager.cc:194] Registered new tserver with Master: 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207)
I20260619 12:27:49.024277 23013 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:38803
I20260619 12:27:49.029917 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:27:49.034152 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.030568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44040 (local address 127.6.231.254:34559)
0619 12:27:49.030695 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:49.030700 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:49.030900 (+ 200us) server_negotiation.cc:407] Connection header received
0619 12:27:49.031037 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:49.031039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:49.031152 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:49.031247 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:27:49.031456 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.031962 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.032895 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.033121 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.033203 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.033625 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:27:49.033801 (+ 176us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0619 12:27:49.034053 (+ 252us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44040: token expired
Metrics: {"server-negotiator.queue_time_us":54}
W20260619 12:27:49.034272 23203 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44040: token expired
W20260619 12:27:49.034623 22941 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.6.231.254:34559)
I20260619 12:27:49.042701 23203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.035062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44052 (local address 127.6.231.254:34559)
0619 12:27:49.035200 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:49.035204 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:49.035217 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:27:49.035255 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:49.035257 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:49.035293 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:49.035360 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:27:49.036019 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.036500 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.037340 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.037757 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.037842 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.039286 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:27:49.039289 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:27:49.039291 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:27:49.039313 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:27:49.040932 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:49.041354 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:49.041357 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:49.041358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:49.041525 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:27:49.041854 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:27:49.041857 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:27:49.041858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:27:49.042030 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:27:49.042140 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:49.042443 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 12:27:49.042539 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 12:27:49.045382 23013 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:27:49.048832 23013 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:27:49.057057 23135 tablet_service.cc:1467] Processing CreateTablet for tablet d452c1acdca14a2e9df3a8a1754dd270 (DEFAULT_TABLE table=test-table [id=f4cb3074df044db5bf79648d62f9209b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:49.057819 23135 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d452c1acdca14a2e9df3a8a1754dd270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:49.060662 23093 tablet_bootstrap.cc:492] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72: Bootstrap starting.
I20260619 12:27:49.061609 23093 tablet_bootstrap.cc:654] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:49.062147 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.056269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:40207 (local address 127.0.0.1:47632)
0619 12:27:49.056972 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:49.056983 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:49.057099 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:49.057812 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:49.057817 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:49.057835 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:49.057926 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:49.057931 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.059410 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.059413 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:49.060265 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:49.060274 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.061611 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.061615 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:49.061709 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.061765 (+ 56us) client_negotiation.cc:769] Sending connection context
0619 12:27:49.061871 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 12:27:49.061968 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"mutex_wait_us":25,"thread_start_us":121,"threads_started":1}
I20260619 12:27:49.062184 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.056505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47632 (local address 127.6.231.195:40207)
0619 12:27:49.056860 (+ 355us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:49.056873 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:49.057077 (+ 204us) server_negotiation.cc:407] Connection header received
0619 12:27:49.057213 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:49.057223 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:49.057472 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:49.057630 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:49.058202 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.059312 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.060406 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.061250 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.061356 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.061540 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:49.061922 (+ 382us) server_negotiation.cc:299] Negotiation successful
0619 12:27:49.062028 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":167,"threads_started":1}
I20260619 12:27:49.062674 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.056065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:46563 (local address 127.0.0.1:35736)
0619 12:27:49.056296 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:49.056312 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:49.056394 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:49.057613 (+ 1219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:49.057616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:49.057634 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:49.057752 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:49.057759 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.059368 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.059374 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:49.060573 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:49.060584 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.062283 (+ 1699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.062287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:49.062391 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.062420 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 12:27:49.062481 (+ 61us) client_negotiation.cc:240] Negotiation successful
0619 12:27:49.062554 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":31}
I20260619 12:27:49.063205 23482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.056606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35736 (local address 127.6.231.194:46563)
0619 12:27:49.057050 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:49.057061 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:49.057085 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:27:49.057168 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:49.057174 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:49.057413 (+ 239us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:49.057608 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:49.057988 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.059230 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.061122 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.062120 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.062849 (+ 729us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.062989 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:49.063030 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 12:27:49.063099 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":144,"threads_started":1}
I20260619 12:27:49.063814 23093 tablet_bootstrap.cc:492] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72: No bootstrap required, opened a new log
I20260619 12:27:49.063907 23093 ts_tablet_manager.cc:1397] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260619 12:27:49.064359 23093 raft_consensus.cc:348] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:27:49.064486 23093 raft_consensus.cc:374] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:49.064523 23093 raft_consensus.cc:729] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8539e55d90f8497b8ddf110216e29a72, State: Initialized, Role: FOLLOWER
I20260619 12:27:49.064682 23093 consensus_queue.cc:260] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:27:49.065387 23093 ts_tablet_manager.cc:1428] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260619 12:27:49.065593 23405 tablet_service.cc:1467] Processing CreateTablet for tablet d452c1acdca14a2e9df3a8a1754dd270 (DEFAULT_TABLE table=test-table [id=f4cb3074df044db5bf79648d62f9209b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:49.065989 23273 tablet_service.cc:1467] Processing CreateTablet for tablet d452c1acdca14a2e9df3a8a1754dd270 (DEFAULT_TABLE table=test-table [id=f4cb3074df044db5bf79648d62f9209b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:27:49.066790 23405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d452c1acdca14a2e9df3a8a1754dd270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:49.067000 23273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d452c1acdca14a2e9df3a8a1754dd270. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:27:49.073688 23485 tablet_bootstrap.cc:492] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7: Bootstrap starting.
I20260619 12:27:49.074343 23484 tablet_bootstrap.cc:492] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a: Bootstrap starting.
I20260619 12:27:49.076241 23485 tablet_bootstrap.cc:654] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7: Neither blocks nor log segments found. Creating new log.
W20260619 12:27:49.076368 23202 tablet.cc:2369] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:27:49.076560 23484 tablet_bootstrap.cc:654] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a: Neither blocks nor log segments found. Creating new log.
I20260619 12:27:49.077098 23485 log.cc:824] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:49.077322 23484 log.cc:824] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:27:49.079583 23485 tablet_bootstrap.cc:492] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7: No bootstrap required, opened a new log
I20260619 12:27:49.079797 23485 ts_tablet_manager.cc:1397] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.002s
I20260619 12:27:49.080356 23484 tablet_bootstrap.cc:492] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a: No bootstrap required, opened a new log
I20260619 12:27:49.080538 23484 ts_tablet_manager.cc:1397] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260619 12:27:49.083823 23485 raft_consensus.cc:348] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:27:49.084126 23485 raft_consensus.cc:374] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:49.084177 23485 raft_consensus.cc:729] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3bab1a833f7b41b3ac182d27be5b6af7, State: Initialized, Role: FOLLOWER
I20260619 12:27:49.084300 23484 raft_consensus.cc:348] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:27:49.084542 23484 raft_consensus.cc:374] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:27:49.084602 23484 raft_consensus.cc:729] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 137c44acca1442a4ad9643b25354c80a, State: Initialized, Role: FOLLOWER
I20260619 12:27:49.084652 23485 consensus_queue.cc:260] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:27:49.085000 23484 consensus_queue.cc:260] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:27:49.085390 23339 heartbeater.cc:503] Master 127.6.231.254:34559 was elected leader, sending a full tablet report...
I20260619 12:27:49.085657 23471 heartbeater.cc:503] Master 127.6.231.254:34559 was elected leader, sending a full tablet report...
I20260619 12:27:49.085757 23485 ts_tablet_manager.cc:1428] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260619 12:27:49.085947 23484 ts_tablet_manager.cc:1428] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260619 12:27:49.254043 23472 tablet.cc:2369] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:27:49.308643 23340 tablet.cc:2369] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:27:49.392149 23489 raft_consensus.cc:482] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:27:49.392383 23489 raft_consensus.cc:504] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:27:49.393530 23489 leader_election.cc:288] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563), 8539e55d90f8497b8ddf110216e29a72 (127.6.231.193:35497)
I20260619 12:27:49.398875 23482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.394011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:45003 (local address 127.6.231.194:46563)
0619 12:27:49.394150 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:49.394155 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:49.394174 (+ 19us) server_negotiation.cc:407] Connection header received
0619 12:27:49.394289 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:49.394292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:49.394349 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:49.394442 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:49.394806 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.395667 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.397574 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.398147 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.398319 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.398381 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:49.398643 (+ 262us) server_negotiation.cc:299] Negotiation successful
0619 12:27:49.398739 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 12:27:49.398923 23490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.393985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:49215 (local address 127.6.231.193:35497)
0619 12:27:49.394342 (+ 357us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:49.394347 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:49.394362 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:27:49.394578 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:49.394584 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:49.394648 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:49.394816 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:27:49.395105 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.395843 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.397574 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.398272 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.398378 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.398519 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:49.398643 (+ 124us) server_negotiation.cc:299] Negotiation successful
0619 12:27:49.398798 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":95,"threads_started":1}
I20260619 12:27:49.398950 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.393778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:46563 (local address 127.6.231.195:45003)
0619 12:27:49.394026 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:49.394047 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:49.394160 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:49.394482 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:49.394485 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:49.394503 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:49.394682 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:49.394688 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.395806 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.395809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:49.397408 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:49.397416 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.398343 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.398350 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:49.398482 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.398528 (+ 46us) client_negotiation.cc:769] Sending connection context
0619 12:27:49.398608 (+ 80us) client_negotiation.cc:240] Negotiation successful
0619 12:27:49.398764 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":232}
I20260619 12:27:49.399164 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.393778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:35497 (local address 127.6.231.195:49215)
0619 12:27:49.394313 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:49.394324 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:49.394392 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:49.394835 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:27:49.394837 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:27:49.394853 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:27:49.395005 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:49.395013 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.395946 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.395949 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:49.397405 (+ 1456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:27:49.397416 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.398380 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.398382 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:27:49.398486 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.398533 (+ 47us) client_negotiation.cc:769] Sending connection context
0619 12:27:49.398608 (+ 75us) client_negotiation.cc:240] Negotiation successful
0619 12:27:49.398764 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":311,"thread_start_us":133,"threads_started":1}
I20260619 12:27:49.399734 23293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d452c1acdca14a2e9df3a8a1754dd270" candidate_uuid: "137c44acca1442a4ad9643b25354c80a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" is_pre_election: true
I20260619 12:27:49.399799 23155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d452c1acdca14a2e9df3a8a1754dd270" candidate_uuid: "137c44acca1442a4ad9643b25354c80a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8539e55d90f8497b8ddf110216e29a72" is_pre_election: true
I20260619 12:27:49.400065 23155 raft_consensus.cc:2454] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 137c44acca1442a4ad9643b25354c80a in term 0.
I20260619 12:27:49.400065 23293 raft_consensus.cc:2454] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 137c44acca1442a4ad9643b25354c80a in term 0.
I20260619 12:27:49.400560 23360 leader_election.cc:302] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 137c44acca1442a4ad9643b25354c80a, 3bab1a833f7b41b3ac182d27be5b6af7; no voters:
I20260619 12:27:49.400800 23489 raft_consensus.cc:2781] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:27:49.400904 23489 raft_consensus.cc:482] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:27:49.400966 23489 raft_consensus.cc:3037] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:49.402117 23489 raft_consensus.cc:504] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:27:49.402529 23489 leader_election.cc:288] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [CANDIDATE]: Term 1 election: Requested vote from peers 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563), 8539e55d90f8497b8ddf110216e29a72 (127.6.231.193:35497)
I20260619 12:27:49.402880 23155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d452c1acdca14a2e9df3a8a1754dd270" candidate_uuid: "137c44acca1442a4ad9643b25354c80a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8539e55d90f8497b8ddf110216e29a72"
I20260619 12:27:49.402935 23293 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d452c1acdca14a2e9df3a8a1754dd270" candidate_uuid: "137c44acca1442a4ad9643b25354c80a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3bab1a833f7b41b3ac182d27be5b6af7"
I20260619 12:27:49.403031 23155 raft_consensus.cc:3037] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:49.403069 23293 raft_consensus.cc:3037] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:27:49.404062 23155 raft_consensus.cc:2454] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 137c44acca1442a4ad9643b25354c80a in term 1.
I20260619 12:27:49.404165 23293 raft_consensus.cc:2454] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 137c44acca1442a4ad9643b25354c80a in term 1.
I20260619 12:27:49.404381 23358 leader_election.cc:302] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 137c44acca1442a4ad9643b25354c80a, 8539e55d90f8497b8ddf110216e29a72; no voters:
I20260619 12:27:49.404551 23489 raft_consensus.cc:2781] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:27:49.404784 23489 raft_consensus.cc:686] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [term 1 LEADER]: Becoming Leader. State: Replica: 137c44acca1442a4ad9643b25354c80a, State: Running, Role: LEADER
I20260619 12:27:49.405145 23489 consensus_queue.cc:237] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:27:49.408339 23013 catalog_manager.cc:5495] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a reported cstate change: term changed from 0 to 1, leader changed from <none> to 137c44acca1442a4ad9643b25354c80a (127.6.231.195). New cstate: current_term: 1 leader_uuid: "137c44acca1442a4ad9643b25354c80a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } health_report { overall_health: UNKNOWN } } }
I20260619 12:27:49.437227 23481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:27:49.433802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53998 (local address 127.6.231.195:40207)
0619 12:27:49.433943 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 12:27:49.433953 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:27:49.433966 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:27:49.434085 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:27:49.434087 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:27:49.434141 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:27:49.434241 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:27:49.434467 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.435018 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.435907 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:27:49.436119 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:27:49.436199 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:27:49.436478 (+ 279us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:27:49.436901 (+ 423us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:27:49.436994 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 12:27:49.437047 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 12:27:49.437097 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:27:49.443948 23293 raft_consensus.cc:1261] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 1 FOLLOWER]: Refusing update from remote peer 137c44acca1442a4ad9643b25354c80a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:27:49.443967 23155 raft_consensus.cc:1261] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Refusing update from remote peer 137c44acca1442a4ad9643b25354c80a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:27:49.444751 23489 consensus_queue.cc:1035] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:27:49.444959 23492 consensus_queue.cc:1035] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [LEADER]: Connected to new peer: Peer: permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:27:49.452804 23496 mvcc.cc:205] Tried to move back new op lower bound from 7298547996433747968 to 7298547996285730816. Current Snapshot: MvccSnapshot[applied={T|T < 7298547996433747968}]
I20260619 12:27:49.453023 23495 mvcc.cc:205] Tried to move back new op lower bound from 7298547996433747968 to 7298547996285730816. Current Snapshot: MvccSnapshot[applied={T|T < 7298547996433747968}]
I20260619 12:27:49.459270 23011 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:27:49.459421 23011 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:27:49.461700 23011 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Sending DeleteTablet for 3 replicas of tablet d452c1acdca14a2e9df3a8a1754dd270
I20260619 12:27:49.462386 23273 tablet_service.cc:1514] Processing DeleteTablet for tablet d452c1acdca14a2e9df3a8a1754dd270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:49 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:35736
I20260619 12:27:49.462400 23405 tablet_service.cc:1514] Processing DeleteTablet for tablet d452c1acdca14a2e9df3a8a1754dd270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:49 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:47632
I20260619 12:27:49.462479 23135 tablet_service.cc:1514] Processing DeleteTablet for tablet d452c1acdca14a2e9df3a8a1754dd270 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:27:49 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:38488
I20260619 12:27:49.462909 23506 tablet_replica.cc:330] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7: stopping tablet replica
I20260619 12:27:49.462920 23504 tablet_replica.cc:330] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72: stopping tablet replica
I20260619 12:27:49.463038 23504 raft_consensus.cc:2229] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:27:49.463064 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 23051
I20260619 12:27:49.463812 23505 tablet_replica.cc:330] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a: stopping tablet replica
I20260619 12:27:49.464121 23505 raft_consensus.cc:2229] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:27:49.464497 23505 raft_consensus.cc:2258] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:27:49.466094 23506 raft_consensus.cc:2229] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:27:49.466356 23506 raft_consensus.cc:2258] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:27:49.467598 23506 ts_tablet_manager.cc:1905] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 12:27:49.469595 22997 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:35497 (error 108)
I20260619 12:27:49.470341 23506 ts_tablet_manager.cc:1918] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:27:49.470439 23506 log.cc:1196] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/d452c1acdca14a2e9df3a8a1754dd270
I20260619 12:27:49.470781 23506 ts_tablet_manager.cc:1939] T d452c1acdca14a2e9df3a8a1754dd270 P 3bab1a833f7b41b3ac182d27be5b6af7: Deleting consensus metadata
I20260619 12:27:49.472150 22999 catalog_manager.cc:4841] TS 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563): tablet d452c1acdca14a2e9df3a8a1754dd270 (table test-table [id=f4cb3074df044db5bf79648d62f9209b]) successfully deleted
W20260619 12:27:49.472497 23208 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:27:49.470596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:35497 (local address 127.0.0.1:34634)
0619 12:27:49.470947 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0619 12:27:49.470966 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 12:27:49.471117 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:27:49.472403 (+ 1286us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:35497: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":109}
W20260619 12:27:49.472668 22997 catalog_manager.cc:4584] TS 8539e55d90f8497b8ddf110216e29a72 (127.6.231.193:35497): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d452c1acdca14a2e9df3a8a1754dd270: Network error: Client connection negotiation failed: client connection to 127.6.231.193:35497: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 12:27:49.472990 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 23205
I20260619 12:27:49.473927 23505 ts_tablet_manager.cc:1905] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:27:49.476516 23505 ts_tablet_manager.cc:1918] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:27:49.476615 23505 log.cc:1196] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/d452c1acdca14a2e9df3a8a1754dd270
I20260619 12:27:49.476929 23505 ts_tablet_manager.cc:1939] T d452c1acdca14a2e9df3a8a1754dd270 P 137c44acca1442a4ad9643b25354c80a: Deleting consensus metadata
I20260619 12:27:49.478204 23000 catalog_manager.cc:4841] TS 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207): tablet d452c1acdca14a2e9df3a8a1754dd270 (table test-table [id=f4cb3074df044db5bf79648d62f9209b]) successfully deleted
I20260619 12:27:49.480280 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 23342
I20260619 12:27:49.487000 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 22980
I20260619 12:28:09.493772 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:34559
--webserver_interface=127.6.231.254
--webserver_port=39215
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:34559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:09.609133 23508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:09.609464 23508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:09.609509 23508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:09.613751 23508 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260619 12:28:09.613834 23508 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260619 12:28:09.613852 23508 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260619 12:28:09.613866 23508 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:09.613885 23508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:09.613904 23508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:09.613981 23508 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:09.614001 23508 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:09.620035 23508 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:34559
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:34559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=39215
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23508
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:09.621227 23508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:09.622391 23508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:09.631394 23513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:09.631548 23514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:09.631501 23508 server_base.cc:1034] running on GCE node
W20260619 12:28:09.631501 23516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:09.632414 23508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:09.633857 23508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:09.635080 23508 hybrid_clock.cc:648] HybridClock initialized: now 1781872089635056 us; error 62 us; skew 500 ppm
Jun 19 12:28:09 dist-test-slave-kr9h krb5kdc[22433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872089, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:09.644099 23508 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:28:09.645166 23508 webserver.cc:460] Webserver started at http://127.6.231.254:39215/ using document root <none> and password file <none>
I20260619 12:28:09.645875 23508 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:09.645979 23508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:09.650038 23508 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:28:09.652215 23523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:09.653610 23508 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:28:09.653748 23508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "b0621e85d9924636b0283821c13a5a65"
format_stamp: "Formatted at 2026-06-19 12:27:26 on dist-test-slave-kr9h"
server_key: "62bd105f9d8a84b24b7587aa60c4b10d"
server_key_iv: "41789141532c2933b9cfc071a7c66218"
server_key_version: "encryptionkey@0"
I20260619 12:28:09.654323 23508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:09.668916 23508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:09.672119 23508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:09.672331 23508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:09.679912 23508 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:34559
I20260619 12:28:09.680230 23575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:34559 every 8 connection(s)
I20260619 12:28:09.681108 23508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:09.685614 23576 sys_catalog.cc:263] Verifying existing consensus state
I20260619 12:28:09.687345 23576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Bootstrap starting.
I20260619 12:28:09.690652 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 23508
I20260619 12:28:09.691113 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:35497
--local_ip_for_outbound_sockets=127.6.231.193
--tserver_master_addrs=127.6.231.254:34559
--webserver_port=43409
--webserver_interface=127.6.231.193
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:28:09.704833 23576 log.cc:824] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:09.716207 23576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:28:09.716624 23576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Bootstrap complete.
I20260619 12:28:09.720679 23576 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } }
I20260619 12:28:09.721122 23576 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0621e85d9924636b0283821c13a5a65, State: Initialized, Role: FOLLOWER
I20260619 12:28:09.721674 23576 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } }
I20260619 12:28:09.721836 23576 raft_consensus.cc:388] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:09.721920 23576 raft_consensus.cc:482] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:09.722054 23576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 2 FOLLOWER]: Advancing to term 3
I20260619 12:28:09.724062 23576 raft_consensus.cc:504] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } }
I20260619 12:28:09.724416 23576 leader_election.cc:302] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b0621e85d9924636b0283821c13a5a65; no voters:
I20260619 12:28:09.724767 23576 leader_election.cc:288] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260619 12:28:09.724882 23580 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 3 FOLLOWER]: Leader election won for term 3
I20260619 12:28:09.725085 23580 raft_consensus.cc:686] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [term 3 LEADER]: Becoming Leader. State: Replica: b0621e85d9924636b0283821c13a5a65, State: Running, Role: LEADER
I20260619 12:28:09.725492 23580 consensus_queue.cc:237] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } }
I20260619 12:28:09.726102 23576 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:09.727028 23582 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b0621e85d9924636b0283821c13a5a65. Latest consensus state: current_term: 3 leader_uuid: "b0621e85d9924636b0283821c13a5a65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } } }
I20260619 12:28:09.727147 23582 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:09.727531 23589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:09.727535 23581 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "b0621e85d9924636b0283821c13a5a65" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b0621e85d9924636b0283821c13a5a65" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 34559 } } }
I20260619 12:28:09.727702 23581 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:09.733029 23589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:09.733844 23589 catalog_manager.cc:1259] Loaded cluster ID: 336742e49aef40a9b6d8c62cc8ffb3b9
I20260619 12:28:09.733961 23589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:09.736574 23589 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:09.737417 23589 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Loaded TSK: 0
I20260619 12:28:09.737622 23589 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b0621e85d9924636b0283821c13a5a65: Loaded TSK: 1
F20260619 12:28:09.738507 23589 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7ffa4eb1a9ad google::LogMessage::Fail() at ??:0
@ 0x7ffa4eb1d88a google::LogMessage::SendToLog() at ??:0
@ 0x7ffa4eb1a4de google::LogMessage::Flush() at ??:0
@ 0x7ffa4eb1ba19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7ffa517de18d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7ffa517d6f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7ffa5947f7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7ffa5945999c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7ffa5945a178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7ffa5949bdf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7ffa594ba82b std::function<>::operator()() at ??:0
@ 0x7ffa59459b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7ffa5945af3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7ffa59457579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7ffa5949b640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ffa50d6c3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7ffa50d6cc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7ffa50d6e3f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7ffa50d5d512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7ffa523a16db start_thread at ??:0
@ 0x7ffa4d61861f clone at ??:0
W20260619 12:28:09.840677 23578 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:09.841069 23578 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:09.841141 23578 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:09.847671 23578 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:09.847764 23578 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:09.847875 23578 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:28:09.856360 23578 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:35497
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=43409
--tserver_master_addrs=127.6.231.254:34559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23578
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:09.857947 23578 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:09.859326 23578 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:09.872218 23624 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:09.872259 23626 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:09.872635 23623 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:09.873317 23578 server_base.cc:1034] running on GCE node
I20260619 12:28:09.873858 23578 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:09.874670 23578 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:09.875891 23578 hybrid_clock.cc:648] HybridClock initialized: now 1781872089875866 us; error 36 us; skew 500 ppm
Jun 19 12:28:09 dist-test-slave-kr9h krb5kdc[22433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872089, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:09.879381 23578 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:28:09.880371 23578 webserver.cc:460] Webserver started at http://127.6.231.193:43409/ using document root <none> and password file <none>
I20260619 12:28:09.880954 23578 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:09.881023 23578 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:09.885557 23578 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260619 12:28:09.888171 23633 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:09.889678 23578 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 12:28:09.889817 23578 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "8539e55d90f8497b8ddf110216e29a72"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "7ec15d280fdc80b421d28820c3e746b5"
server_key_iv: "35e21a8d44e4683550bc2db691614469"
server_key_version: "encryptionkey@0"
I20260619 12:28:09.890427 23578 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:09.920819 23578 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:09.924474 23578 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:09.924711 23578 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:09.925340 23578 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:09.926510 23640 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260619 12:28:09.932278 23578 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260619 12:28:09.932371 23578 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260619 12:28:09.932484 23578 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260619 12:28:09.934345 23578 ts_tablet_manager.cc:610] Registered 1 tablets
I20260619 12:28:09.934419 23578 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260619 12:28:09.934736 23640 tablet_bootstrap.cc:492] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72: Bootstrap starting.
I20260619 12:28:09.945164 23578 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:35497
I20260619 12:28:09.945182 23747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:35497 every 8 connection(s)
I20260619 12:28:09.946229 23578 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:28:09.949342 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 23578
I20260619 12:28:09.949822 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:46563
--local_ip_for_outbound_sockets=127.6.231.194
--tserver_master_addrs=127.6.231.254:34559
--webserver_port=46527
--webserver_interface=127.6.231.194
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:28:09.956825 23640 log.cc:824] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:09.966809 23640 tablet_bootstrap.cc:492] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260619 12:28:09.967304 23640 tablet_bootstrap.cc:492] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72: Bootstrap complete.
I20260619 12:28:09.968886 23640 ts_tablet_manager.cc:1397] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72: Time spent bootstrapping tablet: real 0.034s user 0.028s sys 0.005s
I20260619 12:28:09.972297 23640 raft_consensus.cc:348] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:28:09.972826 23640 raft_consensus.cc:729] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8539e55d90f8497b8ddf110216e29a72, State: Initialized, Role: FOLLOWER
I20260619 12:28:09.973359 23640 consensus_queue.cc:260] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:28:09.974445 23640 ts_tablet_manager.cc:1428] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260619 12:28:09.995239 23750 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:28:09.948313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:54545)
0619 12:28:09.948801 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:09.948855 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:09.951311 (+ 2456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:09.994271 (+ 42960us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":143,"threads_started":1}
W20260619 12:28:09.995774 23748 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:34559 (0 consecutive failures): Network error: Failed to ping master at 127.6.231.254:34559: Client connection negotiation failed: client connection to 127.6.231.254:34559: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260619 12:28:09.996657 23750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:09.996309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:39105)
0619 12:28:09.996479 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:09.996570 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260619 12:28:09.997359 23750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:09.997131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:42983)
0619 12:28:09.997250 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:09.997301 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 12:28:09.997534 23748 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260619 12:28:10.067294 23751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:10.067565 23751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:10.067612 23751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:10.071923 23751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:10.071998 23751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:10.072060 23751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:28:10.077347 23751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:46563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=46527
--tserver_master_addrs=127.6.231.254:34559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:10.078481 23751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:10.079573 23751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:10.089344 23761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:10.089533 23751 server_base.cc:1034] running on GCE node
W20260619 12:28:10.089339 23758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:10.089725 23759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:10.090137 23751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:10.090803 23751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:10.091987 23751 hybrid_clock.cc:648] HybridClock initialized: now 1781872090091966 us; error 40 us; skew 500 ppm
Jun 19 12:28:10 dist-test-slave-kr9h krb5kdc[22433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872090, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:10.094743 23751 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:28:10.095609 23751 webserver.cc:460] Webserver started at http://127.6.231.194:46527/ using document root <none> and password file <none>
I20260619 12:28:10.096114 23751 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:10.096172 23751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:10.099959 23751 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:28:10.102140 23768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:10.103379 23751 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:28:10.103482 23751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "3bab1a833f7b41b3ac182d27be5b6af7"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "59e24a1957e60834ad62797829fadbfe"
server_key_iv: "43cd99ee3e148d03d26b2eeca58ad4c0"
server_key_version: "encryptionkey@0"
I20260619 12:28:10.104051 23751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:10.113884 23751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:10.117020 23751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:10.117241 23751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:10.117935 23751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:10.118814 23751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:10.118868 23751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:10.118938 23751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:10.118971 23751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:10.128024 23751 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:46563
I20260619 12:28:10.128049 23881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:46563 every 8 connection(s)
I20260619 12:28:10.129040 23751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:28:10.133009 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:10.131239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:41389)
0619 12:28:10.132141 (+ 902us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:10.132265 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":527,"threads_started":1}
W20260619 12:28:10.133585 23882 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:34559 (0 consecutive failures): Network error: Failed to ping master at 127.6.231.254:34559: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
I20260619 12:28:10.134157 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:10.133955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:56143)
0619 12:28:10.134059 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:10.134105 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260619 12:28:10.134770 23884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:10.134575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:33925)
0619 12:28:10.134676 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:10.134727 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 12:28:10.134928 23882 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:28:10.138098 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 23751
I20260619 12:28:10.138537 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:40207
--local_ip_for_outbound_sockets=127.6.231.195
--tserver_master_addrs=127.6.231.254:34559
--webserver_port=44329
--webserver_interface=127.6.231.195
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:44553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:10.252990 23885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:10.253276 23885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:10.253321 23885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:10.257572 23885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:10.257632 23885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:10.257733 23885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:28:10.263190 23885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:44553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:40207
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=44329
--tserver_master_addrs=127.6.231.254:34559
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:10.264335 23885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:10.265410 23885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:10.275790 23893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:10.275806 23890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:10.275899 23885 server_base.cc:1034] running on GCE node
W20260619 12:28:10.275806 23891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:10.276561 23885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:10.277246 23885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:10.278409 23885 hybrid_clock.cc:648] HybridClock initialized: now 1781872090278377 us; error 62 us; skew 500 ppm
Jun 19 12:28:10 dist-test-slave-kr9h krb5kdc[22433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872090, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:10.281304 23885 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:28:10.282277 23885 webserver.cc:460] Webserver started at http://127.6.231.195:44329/ using document root <none> and password file <none>
I20260619 12:28:10.282859 23885 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:10.282972 23885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:10.287058 23885 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:28:10.289316 23900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:10.290679 23885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:10.290819 23885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "137c44acca1442a4ad9643b25354c80a"
format_stamp: "Formatted at 2026-06-19 12:27:27 on dist-test-slave-kr9h"
server_key: "a166ab4b4322ecf3789a921d74eb7b13"
server_key_iv: "b950219cddf32b05a9557574c7884be0"
server_key_version: "encryptionkey@0"
I20260619 12:28:10.291399 23885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:10.305992 23885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:10.309767 23885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:10.309993 23885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:10.310611 23885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:10.311462 23885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:10.311533 23885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:10.311602 23885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:10.311650 23885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:10.320801 23885 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:40207
I20260619 12:28:10.320884 24013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:40207 every 8 connection(s)
I20260619 12:28:10.321929 23885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:28:10.325289 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 23885
I20260619 12:28:10.325726 24016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:10.324169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:53141)
0619 12:28:10.324825 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:10.324931 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":111,"threads_started":1}
W20260619 12:28:10.326277 24014 heartbeater.cc:644] Failed to heartbeat to 127.6.231.254:34559 (0 consecutive failures): Network error: Failed to ping master at 127.6.231.254:34559: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
I20260619 12:28:10.326853 24016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:10.326659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:49231)
0619 12:28:10.326759 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:10.326804 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260619 12:28:10.327486 24016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:10.327222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:46071)
0619 12:28:10.327348 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:10.327406 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260619 12:28:10.327672 24014 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260619 12:28:10.998757 24018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:10.998270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:55033)
0619 12:28:10.998576 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:10.998641 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":102,"threads_started":1}
I20260619 12:28:11.135934 24019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:11.135498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:40467)
0619 12:28:11.135738 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:11.135797 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":70,"threads_started":1}
I20260619 12:28:11.260483 24020 raft_consensus.cc:482] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:28:11.260695 24020 raft_consensus.cc:504] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:28:11.261940 24020 leader_election.cc:288] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563), 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207)
Jun 19 12:28:11 dist-test-slave-kr9h krb5kdc[22433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872089, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.194@KRBTEST.COM
Jun 19 12:28:11 dist-test-slave-kr9h krb5kdc[22433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872089, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.195@KRBTEST.COM
I20260619 12:28:11.278832 24022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:11.262293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:35629 (local address 127.6.231.194:46563)
0619 12:28:11.262715 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:11.262742 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:11.262789 (+ 47us) server_negotiation.cc:407] Connection header received
0619 12:28:11.263714 (+ 925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:11.263721 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:11.264175 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:11.264526 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:11.266479 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:11.268107 (+ 1628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:11.269851 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:11.270385 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:11.270571 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:11.273963 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:11.273969 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:11.273983 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:11.274013 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:11.276210 (+ 2197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:11.276772 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:11.276780 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:11.276789 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:11.276906 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:11.277257 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:11.277260 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:11.277262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:11.277737 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:11.277877 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:11.278410 (+ 533us) server_negotiation.cc:299] Negotiation successful
0619 12:28:11.278671 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":69,"threads_started":1}
I20260619 12:28:11.278820 24018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:11.262143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:46563 (local address 127.6.231.193:35629)
0619 12:28:11.262332 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:11.262351 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:11.262454 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:11.264670 (+ 2216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:11.264672 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:11.265360 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:11.266283 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:11.266306 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:11.268291 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:11.268295 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:11.269717 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:11.269728 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:11.270519 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:11.270523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:11.270710 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:11.271440 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:28:11.271467 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:11.273731 (+ 2264us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:11.276351 (+ 2620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:11.276357 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:11.276372 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:11.276644 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:11.277009 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:11.277012 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:11.277014 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:11.277155 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:11.277860 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:11.277867 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:11.278106 (+ 239us) client_negotiation.cc:769] Sending connection context
0619 12:28:11.278288 (+ 182us) client_negotiation.cc:240] Negotiation successful
0619 12:28:11.278620 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"mutex_wait_us":170,"spinlock_wait_cycles":17792}
I20260619 12:28:11.278957 24021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:11.262222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:40207 (local address 127.6.231.193:59959)
0619 12:28:11.262595 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:11.262612 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:11.262713 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:11.264660 (+ 1947us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:11.264667 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:11.265360 (+ 693us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:11.266283 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:11.266306 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:11.268484 (+ 2178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:11.268487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:11.269883 (+ 1396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:11.269891 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:11.270968 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:11.270972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:11.271067 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:11.271902 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:28:11.271918 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:11.274366 (+ 2448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:11.276806 (+ 2440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:11.276809 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:11.276812 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:11.277029 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:11.277421 (+ 392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:11.277426 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:11.277446 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:11.277541 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:11.278242 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:11.278245 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:11.278345 (+ 100us) client_negotiation.cc:769] Sending connection context
0619 12:28:11.278468 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 12:28:11.278735 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"mutex_wait_us":565,"spinlock_wait_cycles":15104,"thread_start_us":70,"threads_started":1}
I20260619 12:28:11.279687 24023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:11.262432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:59959 (local address 127.6.231.195:40207)
0619 12:28:11.262763 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:11.262773 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:11.262813 (+ 40us) server_negotiation.cc:407] Connection header received
0619 12:28:11.263709 (+ 896us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:11.263717 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:11.264175 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:11.264526 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:11.266479 (+ 1953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:11.268349 (+ 1870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:11.270013 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:11.270844 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:11.271023 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:11.274588 (+ 3565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:11.274598 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:11.274618 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:11.274661 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:11.276651 (+ 1990us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:11.277168 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:11.277174 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:11.277180 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:11.277311 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:11.277660 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:11.277664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:11.277666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:11.278107 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:11.278269 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:11.278650 (+ 381us) server_negotiation.cc:299] Negotiation successful
0619 12:28:11.279547 (+ 897us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222,"thread_start_us":123,"threads_started":1}
W20260619 12:28:11.281781 23637 leader_election.cc:341] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207): Not found: Tablet not found: d452c1acdca14a2e9df3a8a1754dd270
W20260619 12:28:11.281992 23636 leader_election.cc:341] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563): Not found: Tablet not found: d452c1acdca14a2e9df3a8a1754dd270
I20260619 12:28:11.282094 23636 leader_election.cc:302] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8539e55d90f8497b8ddf110216e29a72; no voters: 137c44acca1442a4ad9643b25354c80a, 3bab1a833f7b41b3ac182d27be5b6af7
I20260619 12:28:11.282408 24020 raft_consensus.cc:2726] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:28:11.328677 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:11.328225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:34493)
0619 12:28:11.328502 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:11.328582 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":89,"threads_started":1}
I20260619 12:28:11.999979 24025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:11.999535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:58017)
0619 12:28:11.999818 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:11.999882 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":82,"threads_started":1}
I20260619 12:28:12.137113 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:12.136681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:35727)
0619 12:28:12.136955 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:12.137010 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":107,"threads_started":1}
I20260619 12:28:12.329931 24027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:12.329501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:52313)
0619 12:28:12.329740 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:12.329824 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":72,"threads_started":1}
I20260619 12:28:13.001466 24028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:13.000973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:33659)
0619 12:28:13.001234 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:13.001296 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":77,"threads_started":1}
I20260619 12:28:13.138303 24029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:13.137910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:40807)
0619 12:28:13.138151 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:13.138208 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":71,"threads_started":1}
I20260619 12:28:13.331164 24030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:13.330735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:33239)
0619 12:28:13.331003 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:13.331067 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":79,"threads_started":1}
I20260619 12:28:13.400830 24031 raft_consensus.cc:482] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:28:13.400969 24031 raft_consensus.cc:504] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:28:13.401396 24031 leader_election.cc:288] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563), 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207)
W20260619 12:28:13.402288 23636 leader_election.cc:341] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563): Not found: Tablet not found: d452c1acdca14a2e9df3a8a1754dd270
W20260619 12:28:13.402436 23637 leader_election.cc:341] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207): Not found: Tablet not found: d452c1acdca14a2e9df3a8a1754dd270
I20260619 12:28:13.402565 23637 leader_election.cc:302] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8539e55d90f8497b8ddf110216e29a72; no voters: 137c44acca1442a4ad9643b25354c80a, 3bab1a833f7b41b3ac182d27be5b6af7
I20260619 12:28:13.402730 24031 raft_consensus.cc:2726] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:28:14.002692 24032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:14.002264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:55021)
0619 12:28:14.002528 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:14.002590 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":74,"threads_started":1}
I20260619 12:28:14.139660 24033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:14.139209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:33795)
0619 12:28:14.139485 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:14.139561 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":81,"threads_started":1}
I20260619 12:28:14.332401 24034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:14.331947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:58365)
0619 12:28:14.332238 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:14.332307 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":73,"threads_started":1}
I20260619 12:28:15.004086 24035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:15.003573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:47617)
0619 12:28:15.003896 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:15.003957 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":74,"threads_started":1}
I20260619 12:28:15.041872 24036 raft_consensus.cc:482] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:28:15.042029 24036 raft_consensus.cc:504] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:28:15.042537 24036 leader_election.cc:288] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563), 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207)
W20260619 12:28:15.043483 23636 leader_election.cc:341] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563): Not found: Tablet not found: d452c1acdca14a2e9df3a8a1754dd270
W20260619 12:28:15.043578 23637 leader_election.cc:341] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207): Not found: Tablet not found: d452c1acdca14a2e9df3a8a1754dd270
I20260619 12:28:15.043638 23636 leader_election.cc:302] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8539e55d90f8497b8ddf110216e29a72; no voters: 137c44acca1442a4ad9643b25354c80a, 3bab1a833f7b41b3ac182d27be5b6af7
I20260619 12:28:15.043804 24036 raft_consensus.cc:2726] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:28:15.141175 24037 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:15.140710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:41725)
0619 12:28:15.140982 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:15.141041 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":98,"threads_started":1}
I20260619 12:28:15.333853 24038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:15.333349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:56479)
0619 12:28:15.333689 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:15.333755 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":143,"threads_started":1}
I20260619 12:28:16.005347 24039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:16.004925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:44287)
0619 12:28:16.005188 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:16.005250 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":91,"threads_started":1}
I20260619 12:28:16.142459 24040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:16.141984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:49579)
0619 12:28:16.142273 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:16.142339 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":82,"threads_started":1}
I20260619 12:28:16.335217 24041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:16.334750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:46095)
0619 12:28:16.335035 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:16.335099 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":90,"threads_started":1}
I20260619 12:28:17.006672 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:17.006200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:43947)
0619 12:28:17.006501 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:17.006571 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":87,"threads_started":1}
I20260619 12:28:17.143682 24043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:17.143248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:35945)
0619 12:28:17.143532 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:17.143589 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":73,"threads_started":1}
I20260619 12:28:17.336632 24044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:17.336130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:52995)
0619 12:28:17.336467 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:17.336536 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":94,"threads_started":1}
I20260619 12:28:17.685897 24045 raft_consensus.cc:482] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:28:17.686036 24045 raft_consensus.cc:504] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:28:17.686525 24045 leader_election.cc:288] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563), 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207)
W20260619 12:28:17.687613 23637 leader_election.cc:341] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207): Not found: Tablet not found: d452c1acdca14a2e9df3a8a1754dd270
W20260619 12:28:17.687750 23636 leader_election.cc:341] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563): Not found: Tablet not found: d452c1acdca14a2e9df3a8a1754dd270
I20260619 12:28:17.687959 23636 leader_election.cc:302] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8539e55d90f8497b8ddf110216e29a72; no voters: 137c44acca1442a4ad9643b25354c80a, 3bab1a833f7b41b3ac182d27be5b6af7
I20260619 12:28:17.688220 24045 raft_consensus.cc:2726] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:28:18.008159 24046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:18.007541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:40229)
0619 12:28:18.007972 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:18.008055 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":98,"threads_started":1}
I20260619 12:28:18.145128 24047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:18.144700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:33473)
0619 12:28:18.144967 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:18.145034 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":77,"threads_started":1}
I20260619 12:28:18.338105 24048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:18.337608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:36639)
0619 12:28:18.337922 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:18.337995 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":88,"threads_started":1}
I20260619 12:28:19.009575 24049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:19.009004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:48349)
0619 12:28:19.009303 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:19.009390 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":78,"threads_started":1}
I20260619 12:28:19.146322 24050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:19.145930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:41115)
0619 12:28:19.146174 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:19.146228 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
I20260619 12:28:19.339650 24051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:19.339149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:54137)
0619 12:28:19.339483 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:19.339554 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":104,"threads_started":1}
I20260619 12:28:20.011143 24052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:20.010586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:52933)
0619 12:28:20.010966 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:20.011026 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":90,"threads_started":1}
I20260619 12:28:20.147675 24053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:20.147202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:44505)
0619 12:28:20.147495 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:20.147573 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":90,"threads_started":1}
I20260619 12:28:20.341184 24054 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:20.340654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:58037)
0619 12:28:20.340955 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:20.341082 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":92,"threads_started":1}
I20260619 12:28:21.012498 24055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:21.012060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:35981)
0619 12:28:21.012327 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:21.012398 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":86,"threads_started":1}
I20260619 12:28:21.149055 24056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:21.148590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:38353)
0619 12:28:21.148893 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:21.148957 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":77,"threads_started":1}
I20260619 12:28:21.342507 24057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:21.341997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:52173)
0619 12:28:21.342332 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:21.342410 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":91,"threads_started":1}
I20260619 12:28:22.014003 24058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:22.013541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:54523)
0619 12:28:22.013833 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:22.013900 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":77,"threads_started":1}
I20260619 12:28:22.150617 24059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:22.150040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:60337)
0619 12:28:22.150406 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:22.150500 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":110,"threads_started":1}
I20260619 12:28:22.343870 24060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:22.343437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:60407)
0619 12:28:22.343697 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:22.343764 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":77,"threads_started":1}
I20260619 12:28:22.682708 24061 raft_consensus.cc:482] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:28:22.682890 24061 raft_consensus.cc:504] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3bab1a833f7b41b3ac182d27be5b6af7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 46563 } } peers { permanent_uuid: "137c44acca1442a4ad9643b25354c80a" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40207 } } peers { permanent_uuid: "8539e55d90f8497b8ddf110216e29a72" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35497 } }
I20260619 12:28:22.683537 24061 leader_election.cc:288] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563), 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207)
W20260619 12:28:22.684471 23636 leader_election.cc:341] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 3bab1a833f7b41b3ac182d27be5b6af7 (127.6.231.194:46563): Not found: Tablet not found: d452c1acdca14a2e9df3a8a1754dd270
W20260619 12:28:22.684573 23637 leader_election.cc:341] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 137c44acca1442a4ad9643b25354c80a (127.6.231.195:40207): Not found: Tablet not found: d452c1acdca14a2e9df3a8a1754dd270
I20260619 12:28:22.684697 23637 leader_election.cc:302] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 8539e55d90f8497b8ddf110216e29a72; no voters: 137c44acca1442a4ad9643b25354c80a, 3bab1a833f7b41b3ac182d27be5b6af7
I20260619 12:28:22.684928 24061 raft_consensus.cc:2726] T d452c1acdca14a2e9df3a8a1754dd270 P 8539e55d90f8497b8ddf110216e29a72 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260619 12:28:23.015406 24062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:23.014965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:58593)
0619 12:28:23.015235 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:23.015304 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":78,"threads_started":1}
I20260619 12:28:23.151921 24063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:23.151453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:56215)
0619 12:28:23.151738 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:23.151800 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":76,"threads_started":1}
I20260619 12:28:23.345275 24064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:23.344817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:42867)
0619 12:28:23.345107 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:23.345175 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":79,"threads_started":1}
I20260619 12:28:24.016651 24065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:24.016178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:60623)
0619 12:28:24.016486 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:24.016545 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":133,"threads_started":1}
I20260619 12:28:24.153398 24066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:24.152950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:36693)
0619 12:28:24.153235 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:24.153299 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":81,"threads_started":1}
I20260619 12:28:24.346711 24067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:24.346201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.195:57203)
0619 12:28:24.346541 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:24.346611 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":81,"threads_started":1}
I20260619 12:28:25.018121 24068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:25.017682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.193:35483)
0619 12:28:25.017945 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:25.018009 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":71,"threads_started":1}
I20260619 12:28:25.154980 24069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:25.154499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:34559 (local address 127.6.231.194:37865)
0619 12:28:25.154807 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:25.154878 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.254:34559: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":80,"threads_started":1}
I20260619 12:28:25.334846 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 23578
I20260619 12:28:25.340842 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 23751
I20260619 12:28:25.346884 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 23885
2026-06-19T12:28:25Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59725 ms)
[----------] 30 tests from SecurityITest (248146 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:25 dist-test-slave-kr9h krb5kdc[24072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:25 dist-test-slave-kr9h krb5kdc[24072](info): set up 2 sockets
Jun 19 12:28:25 dist-test-slave-kr9h krb5kdc[24072](info): commencing operation
krb5kdc: starting...
W20260619 12:28:26.413900 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.044s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:26 dist-test-slave-kr9h krb5kdc[24072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:28:26.544770 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:35917
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:39145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:35917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:26.662076 24085 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:26.662374 24085 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:26.662416 24085 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:26.666625 24085 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:26.666703 24085 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:26.666723 24085 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:26.666740 24085 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:26.666756 24085 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:26.672530 24085 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:39145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:35917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:35917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24085
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:26.673730 24085 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:26.674822 24085 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:28:26.683787 24085 server_base.cc:1034] running on GCE node
W20260619 12:28:26.683799 24093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:26.683995 24091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:26.683813 24090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:26.684724 24085 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:26.686180 24085 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:26.687402 24085 hybrid_clock.cc:648] HybridClock initialized: now 1781872106687366 us; error 55 us; skew 500 ppm
Jun 19 12:28:26 dist-test-slave-kr9h krb5kdc[24072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872106, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:26.690590 24085 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:28:26.691759 24085 webserver.cc:460] Webserver started at http://127.6.231.254:33533/ using document root <none> and password file <none>
I20260619 12:28:26.692379 24085 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:26.692481 24085 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:26.692734 24085 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:26.695071 24085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "dd5f86e2041e4b699c90b58146e7035b"
format_stamp: "Formatted at 2026-06-19 12:28:26 on dist-test-slave-kr9h"
server_key: "7bca60f82fe5bb507b420b1ef3833fbb"
server_key_iv: "800f68fd5712ba01520e755021bc8480"
server_key_version: "encryptionkey@0"
I20260619 12:28:26.695732 24085 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "dd5f86e2041e4b699c90b58146e7035b"
format_stamp: "Formatted at 2026-06-19 12:28:26 on dist-test-slave-kr9h"
server_key: "7bca60f82fe5bb507b420b1ef3833fbb"
server_key_iv: "800f68fd5712ba01520e755021bc8480"
server_key_version: "encryptionkey@0"
I20260619 12:28:26.699869 24085 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:28:26.702651 24100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:26.704037 24085 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:26.704231 24085 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "dd5f86e2041e4b699c90b58146e7035b"
format_stamp: "Formatted at 2026-06-19 12:28:26 on dist-test-slave-kr9h"
server_key: "7bca60f82fe5bb507b420b1ef3833fbb"
server_key_iv: "800f68fd5712ba01520e755021bc8480"
server_key_version: "encryptionkey@0"
I20260619 12:28:26.704406 24085 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:26.722038 24085 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:26.725785 24085 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:26.726028 24085 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:26.734395 24085 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:35917
I20260619 12:28:26.734412 24152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:35917 every 8 connection(s)
I20260619 12:28:26.735412 24085 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:26.738732 24153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:26.742053 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 24085
I20260619 12:28:26.742203 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:26.742520 7071 external_mini_cluster.cc:1428] Setting key 51e04ad205cf917a51682134d9a91591
I20260619 12:28:26.745113 24153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b: Bootstrap starting.
I20260619 12:28:26.747453 24153 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:26.748231 24153 log.cc:824] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:26.750347 24153 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b: No bootstrap required, opened a new log
Jun 19 12:28:26 dist-test-slave-kr9h krb5kdc[24072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872106, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:28:26.757642 24153 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5f86e2041e4b699c90b58146e7035b" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35917 } }
I20260619 12:28:26.757956 24153 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:26.758040 24153 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd5f86e2041e4b699c90b58146e7035b, State: Initialized, Role: FOLLOWER
I20260619 12:28:26.758538 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:26.744165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43128 (local address 127.6.231.254:35917)
0619 12:28:26.744663 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:26.744680 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:26.744721 (+ 41us) server_negotiation.cc:407] Connection header received
0619 12:28:26.745558 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:26.745579 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:26.745963 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:26.746358 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:26.747244 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:26.748502 (+ 1258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:26.749489 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:26.750037 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:26.750230 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:26.753044 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:26.753050 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:26.753067 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:26.753100 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:26.755096 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:26.755631 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:26.755638 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:26.755651 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:26.755782 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:26.756116 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:26.756125 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:26.756126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:26.756591 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:26.756828 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:26.757153 (+ 325us) server_negotiation.cc:299] Negotiation successful
0619 12:28:26.757568 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":145,"threads_started":1}
I20260619 12:28:26.758559 24153 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5f86e2041e4b699c90b58146e7035b" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35917 } }
I20260619 12:28:26.758801 24153 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:26.758889 24153 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:26.758994 24153 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:26.760213 24153 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5f86e2041e4b699c90b58146e7035b" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35917 } }
I20260619 12:28:26.760581 24153 leader_election.cc:302] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd5f86e2041e4b699c90b58146e7035b; no voters:
I20260619 12:28:26.760989 24153 leader_election.cc:288] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:26.761117 24158 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:26.761379 24158 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [term 1 LEADER]: Becoming Leader. State: Replica: dd5f86e2041e4b699c90b58146e7035b, State: Running, Role: LEADER
I20260619 12:28:26.761806 24158 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5f86e2041e4b699c90b58146e7035b" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35917 } }
I20260619 12:28:26.762351 24153 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:26.763758 24160 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd5f86e2041e4b699c90b58146e7035b. Latest consensus state: current_term: 1 leader_uuid: "dd5f86e2041e4b699c90b58146e7035b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5f86e2041e4b699c90b58146e7035b" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35917 } } }
I20260619 12:28:26.763710 24159 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd5f86e2041e4b699c90b58146e7035b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd5f86e2041e4b699c90b58146e7035b" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35917 } } }
I20260619 12:28:26.763860 24159 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:26.763860 24160 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b [sys.catalog]: This master's current role is: LEADER
W20260619 12:28:26.767369 24173 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:28:26.767441 24173 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:28:26.767527 24168 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:26.768247 24168 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:26.774268 24168 catalog_manager.cc:1347] Generated new cluster ID: e93465cebf2f477aa088a1f3a8ccc4a1
I20260619 12:28:26.774351 24168 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:26.785573 24168 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:26.787027 24168 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:26.797899 24168 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b: Generated new TSK 0
I20260619 12:28:26.798880 24168 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:28:26.829959 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35917
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:39145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:26.948733 24179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:26.949035 24179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:26.949081 24179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:26.953325 24179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:26.953415 24179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:26.953553 24179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:28:26.959193 24179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:39145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:26.960300 24179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:26.961310 24179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:26.972035 24187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:26.972052 24179 server_base.cc:1034] running on GCE node
W20260619 12:28:26.971997 24184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:26.971997 24185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:26.972689 24179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:26.973397 24179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:26.974613 24179 hybrid_clock.cc:648] HybridClock initialized: now 1781872106974577 us; error 54 us; skew 500 ppm
Jun 19 12:28:26 dist-test-slave-kr9h krb5kdc[24072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872106, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:26.977401 24179 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:28:26.978384 24179 webserver.cc:460] Webserver started at http://127.6.231.193:39631/ using document root <none> and password file <none>
I20260619 12:28:26.979000 24179 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:26.979094 24179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:26.979336 24179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:26.981541 24179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "c510349fb1a94b0280675820197661f5"
format_stamp: "Formatted at 2026-06-19 12:28:26 on dist-test-slave-kr9h"
server_key: "4c89850c00033dd2975d40d62e5a4e65"
server_key_iv: "f639f2f55d63556f22310229d92eeb92"
server_key_version: "encryptionkey@0"
I20260619 12:28:26.982126 24179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "c510349fb1a94b0280675820197661f5"
format_stamp: "Formatted at 2026-06-19 12:28:26 on dist-test-slave-kr9h"
server_key: "4c89850c00033dd2975d40d62e5a4e65"
server_key_iv: "f639f2f55d63556f22310229d92eeb92"
server_key_version: "encryptionkey@0"
I20260619 12:28:26.986119 24179 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260619 12:28:26.988725 24194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:26.990191 24179 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:26.990347 24179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "c510349fb1a94b0280675820197661f5"
format_stamp: "Formatted at 2026-06-19 12:28:26 on dist-test-slave-kr9h"
server_key: "4c89850c00033dd2975d40d62e5a4e65"
server_key_iv: "f639f2f55d63556f22310229d92eeb92"
server_key_version: "encryptionkey@0"
I20260619 12:28:26.990480 24179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:27.018009 24179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:27.021790 24179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:27.022033 24179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:27.022708 24179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:27.023603 24179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:27.023689 24179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:27.023766 24179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:27.023825 24179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:27.033663 24179 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:36111
I20260619 12:28:27.033730 24307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:36111 every 8 connection(s)
I20260619 12:28:27.034710 24179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:28:27.037106 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 24179
I20260619 12:28:27.037367 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:28:27.037736 7071 external_mini_cluster.cc:1428] Setting key 66a3af262a2917f8bd776afc0470644f
Jun 19 12:28:27 dist-test-slave-kr9h krb5kdc[24072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872106, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
I20260619 12:28:27.050575 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.036687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:42199 (local address 127.6.231.254:35917)
0619 12:28:27.036892 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:27.036897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:27.038036 (+ 1139us) server_negotiation.cc:407] Connection header received
0619 12:28:27.039133 (+ 1097us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:27.039136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:27.039187 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:27.039265 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:27.040528 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.041080 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.042854 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.043086 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.043215 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.046424 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:27.046428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:27.046430 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:27.046457 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:27.048059 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:27.048645 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:27.048648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:27.048650 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:27.048733 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:27.049196 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:27.049199 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:27.049201 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:27.049413 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:27.049582 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:27.050200 (+ 618us) server_negotiation.cc:299] Negotiation successful
0619 12:28:27.050344 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:28:27.051646 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.037228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35917 (local address 127.6.231.193:42199)
0619 12:28:27.037810 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:27.037883 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:27.038913 (+ 1030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:27.039442 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:27.039449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:27.039855 (+ 406us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:27.040352 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.040379 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.041235 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.041237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.042715 (+ 1478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.042730 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.043270 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.043278 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.043530 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.044222 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:28:27.044251 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:27.046216 (+ 1965us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:27.048205 (+ 1989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:27.048213 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:27.048233 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:27.048519 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:27.048906 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:27.048909 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:27.048911 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:27.049082 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:27.049582 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:27.049600 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:27.049919 (+ 319us) client_negotiation.cc:769] Sending connection context
0619 12:28:27.050140 (+ 221us) client_negotiation.cc:240] Negotiation successful
0619 12:28:27.050515 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":171,"threads_started":1}
I20260619 12:28:27.053223 24308 heartbeater.cc:347] Connected to a master server at 127.6.231.254:35917
I20260619 12:28:27.053546 24308 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:27.054226 24308 heartbeater.cc:511] Master 127.6.231.254:35917 requested a full tablet report, sending...
I20260619 12:28:27.056164 24117 ts_manager.cc:194] Registered new tserver with Master: c510349fb1a94b0280675820197661f5 (127.6.231.193:36111)
I20260619 12:28:27.058063 24117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:42199
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:28:27.070252 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35917
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:39145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:27.188175 24313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:27.188472 24313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:27.188522 24313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:27.192972 24313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:27.193063 24313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:27.193135 24313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:28:27.199100 24313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:39145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:27.200343 24313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:27.201470 24313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:27.212113 24319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:27.212146 24318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:27.212426 24313 server_base.cc:1034] running on GCE node
W20260619 12:28:27.212113 24321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:27.212985 24313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:27.213805 24313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:27.215023 24313 hybrid_clock.cc:648] HybridClock initialized: now 1781872107215003 us; error 33 us; skew 500 ppm
Jun 19 12:28:27 dist-test-slave-kr9h krb5kdc[24072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872107, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:27.218254 24313 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:28:27.219190 24313 webserver.cc:460] Webserver started at http://127.6.231.194:39911/ using document root <none> and password file <none>
I20260619 12:28:27.219779 24313 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:27.219842 24313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:27.220017 24313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:27.222361 24313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "4aef5ac0c3de46a589744579c562da2a"
format_stamp: "Formatted at 2026-06-19 12:28:27 on dist-test-slave-kr9h"
server_key: "3e5e16cad92cbe1e2ab072c92356d61e"
server_key_iv: "597297b2499f88a3d3e45e2374a1d8ea"
server_key_version: "encryptionkey@0"
I20260619 12:28:27.222944 24313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "4aef5ac0c3de46a589744579c562da2a"
format_stamp: "Formatted at 2026-06-19 12:28:27 on dist-test-slave-kr9h"
server_key: "3e5e16cad92cbe1e2ab072c92356d61e"
server_key_iv: "597297b2499f88a3d3e45e2374a1d8ea"
server_key_version: "encryptionkey@0"
I20260619 12:28:27.227298 24313 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:28:27.230131 24328 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:27.231818 24313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:27.231990 24313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "4aef5ac0c3de46a589744579c562da2a"
format_stamp: "Formatted at 2026-06-19 12:28:27 on dist-test-slave-kr9h"
server_key: "3e5e16cad92cbe1e2ab072c92356d61e"
server_key_iv: "597297b2499f88a3d3e45e2374a1d8ea"
server_key_version: "encryptionkey@0"
I20260619 12:28:27.232158 24313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:27.250625 24313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:27.254359 24313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:27.254587 24313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:27.255232 24313 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:27.256135 24313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:27.256208 24313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:27.256296 24313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:27.256347 24313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:27.266475 24313 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:37759
I20260619 12:28:27.266503 24441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:37759 every 8 connection(s)
I20260619 12:28:27.267491 24313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:28:27.277758 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 24313
Jun 19 12:28:27 dist-test-slave-kr9h krb5kdc[24072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872107, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:28:27.277899 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:28:27.278245 7071 external_mini_cluster.cc:1428] Setting key 14743ce0f3069434009a58e3097cfc34
I20260619 12:28:27.282900 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.269577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:37095 (local address 127.6.231.254:35917)
0619 12:28:27.269731 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:27.269736 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:27.270720 (+ 984us) server_negotiation.cc:407] Connection header received
0619 12:28:27.271625 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:27.271628 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:27.271675 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:27.271757 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:27.273022 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.273676 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.274975 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.275294 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.275409 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.278585 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:27.278589 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:27.278591 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:27.278619 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:27.280302 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:27.280898 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:27.280902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:27.280904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:27.280991 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:27.281505 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:27.281508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:27.281510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:27.281729 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:27.281815 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:27.282595 (+ 780us) server_negotiation.cc:299] Negotiation successful
0619 12:28:27.282771 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:28:27.284193 24444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.270037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35917 (local address 127.6.231.194:37095)
0619 12:28:27.270510 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:27.270562 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:27.271414 (+ 852us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:27.271926 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:27.271933 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:27.272314 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:27.272858 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.272872 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.273858 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.273861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.274825 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.274835 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.275603 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.275607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.275798 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.276572 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:28:27.276591 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:27.278369 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:27.280463 (+ 2094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:27.280470 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:27.280496 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:27.280783 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:27.281097 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:27.281100 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:27.281102 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:27.281342 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:27.281846 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:27.281855 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:27.282303 (+ 448us) client_negotiation.cc:769] Sending connection context
0619 12:28:27.282552 (+ 249us) client_negotiation.cc:240] Negotiation successful
0619 12:28:27.282952 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":109,"threads_started":1}
I20260619 12:28:27.285759 24442 heartbeater.cc:347] Connected to a master server at 127.6.231.254:35917
I20260619 12:28:27.286063 24442 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:27.286602 24442 heartbeater.cc:511] Master 127.6.231.254:35917 requested a full tablet report, sending...
I20260619 12:28:27.287782 24117 ts_manager.cc:194] Registered new tserver with Master: 4aef5ac0c3de46a589744579c562da2a (127.6.231.194:37759)
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
I20260619 12:28:27.288435 24117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:37095
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:28:27.309239 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35917
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:39145
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:27.437932 24447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:27.438290 24447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:27.438388 24447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:27.442842 24447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:27.442950 24447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:27.443046 24447 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:28:27.448887 24447 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:39145
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:27.450328 24447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:27.451453 24447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:27.462005 24452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:27.462059 24455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:27.462054 24447 server_base.cc:1034] running on GCE node
W20260619 12:28:27.462189 24453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:27.462607 24447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:27.463487 24447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:27.464757 24447 hybrid_clock.cc:648] HybridClock initialized: now 1781872107464730 us; error 37 us; skew 500 ppm
Jun 19 12:28:27 dist-test-slave-kr9h krb5kdc[24072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872107, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:27.468011 24447 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:28:27.469048 24447 webserver.cc:460] Webserver started at http://127.6.231.195:37313/ using document root <none> and password file <none>
I20260619 12:28:27.469769 24447 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:27.469835 24447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:27.470055 24447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:27.472321 24447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "2bd5af8dee81450297ebd6c9dafa28f5"
format_stamp: "Formatted at 2026-06-19 12:28:27 on dist-test-slave-kr9h"
server_key: "3f406300fe3f28a6af4c7dccfa66325c"
server_key_iv: "24553a704d606a226bb8f5bf6c0eb99d"
server_key_version: "encryptionkey@0"
I20260619 12:28:27.472936 24447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "2bd5af8dee81450297ebd6c9dafa28f5"
format_stamp: "Formatted at 2026-06-19 12:28:27 on dist-test-slave-kr9h"
server_key: "3f406300fe3f28a6af4c7dccfa66325c"
server_key_iv: "24553a704d606a226bb8f5bf6c0eb99d"
server_key_version: "encryptionkey@0"
I20260619 12:28:27.477092 24447 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 12:28:27.479975 24462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:27.481829 24447 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:27.481984 24447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "2bd5af8dee81450297ebd6c9dafa28f5"
format_stamp: "Formatted at 2026-06-19 12:28:27 on dist-test-slave-kr9h"
server_key: "3f406300fe3f28a6af4c7dccfa66325c"
server_key_iv: "24553a704d606a226bb8f5bf6c0eb99d"
server_key_version: "encryptionkey@0"
I20260619 12:28:27.482117 24447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:27.508571 24447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:27.512020 24447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:27.512255 24447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:27.512917 24447 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:27.513917 24447 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:27.513995 24447 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:27.514071 24447 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:27.514122 24447 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:27.523890 24447 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:45825
I20260619 12:28:27.523922 24575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:45825 every 8 connection(s)
I20260619 12:28:27.524986 24447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:28:27.527710 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 24447
I20260619 12:28:27.527871 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:28:27.528295 7071 external_mini_cluster.cc:1428] Setting key 156a492ad415028c856657e6d04c1876
Jun 19 12:28:27 dist-test-slave-kr9h krb5kdc[24072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872107, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:28:27.540321 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.527154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:51253 (local address 127.6.231.254:35917)
0619 12:28:27.527352 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:27.527357 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:27.528412 (+ 1055us) server_negotiation.cc:407] Connection header received
0619 12:28:27.529596 (+ 1184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:27.529602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:27.529733 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:27.529850 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:27.531410 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.532020 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.533272 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.533580 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.533665 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.536344 (+ 2679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:27.536359 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:27.536361 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:27.536390 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:27.538139 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:27.538692 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:27.538750 (+ 58us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:27.538751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:27.538845 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:27.539230 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:27.539233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:27.539235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:27.539430 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:27.539528 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:27.540072 (+ 544us) server_negotiation.cc:299] Negotiation successful
0619 12:28:27.540191 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260619 12:28:27.541496 24578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.527580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35917 (local address 127.6.231.195:51253)
0619 12:28:27.528143 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:27.528267 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:27.529331 (+ 1064us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:27.530090 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:27.530101 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:27.530644 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:27.531247 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.531262 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.532182 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.532184 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.533142 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.533152 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.533729 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.533733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.533878 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.534457 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:28:27.534472 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:27.536094 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:27.538274 (+ 2180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:27.538280 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:27.538295 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:27.538572 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:27.538955 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:27.538957 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:27.538959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:27.539116 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:27.539536 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:27.539547 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:27.539829 (+ 282us) client_negotiation.cc:769] Sending connection context
0619 12:28:27.539998 (+ 169us) client_negotiation.cc:240] Negotiation successful
0619 12:28:27.540292 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":164,"threads_started":1}
I20260619 12:28:27.543042 24576 heartbeater.cc:347] Connected to a master server at 127.6.231.254:35917
I20260619 12:28:27.543279 24576 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:27.543761 24576 heartbeater.cc:511] Master 127.6.231.254:35917 requested a full tablet report, sending...
I20260619 12:28:27.544956 24117 ts_manager.cc:194] Registered new tserver with Master: 2bd5af8dee81450297ebd6c9dafa28f5 (127.6.231.195:45825)
I20260619 12:28:27.545706 24117 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:51253
I20260619 12:28:27.555416 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:28:27.566231 24156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.557685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43132 (local address 127.6.231.254:35917)
0619 12:28:27.557842 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:27.557846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:27.557951 (+ 105us) server_negotiation.cc:407] Connection header received
0619 12:28:27.558061 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:27.558063 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:27.558107 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:27.558186 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:27.558995 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.559585 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.560578 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.560828 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.560919 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.562678 (+ 1759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:27.562681 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:27.562684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:27.562711 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:27.564413 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:27.564845 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:27.564848 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:27.564849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:27.564925 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:27.565229 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:27.565233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:27.565234 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:27.565565 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:27.565695 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:27.565987 (+ 292us) server_negotiation.cc:299] Negotiation successful
0619 12:28:27.566095 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:28:27.569227 24117 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:28:27.572414 24117 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:28:27.588781 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.582063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54570 (local address 127.6.231.193:36111)
0619 12:28:27.582368 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:27.582376 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:27.582878 (+ 502us) server_negotiation.cc:407] Connection header received
0619 12:28:27.583098 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:27.583105 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:27.583357 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:27.583529 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:27.584031 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.585295 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.587272 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.587951 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.588094 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.588268 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:27.588499 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 12:28:27.588609 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260619 12:28:27.588773 24588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.581907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:36111 (local address 127.0.0.1:54570)
0619 12:28:27.582719 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:27.582782 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:27.582933 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:27.583711 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:27.583719 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:27.583742 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:27.583897 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.583904 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.585462 (+ 1558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.585465 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.587128 (+ 1663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.587137 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.588091 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.588095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.588259 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.588311 (+ 52us) client_negotiation.cc:769] Sending connection context
0619 12:28:27.588460 (+ 149us) client_negotiation.cc:240] Negotiation successful
0619 12:28:27.588603 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"mutex_wait_us":36,"thread_start_us":97,"threads_started":1}
I20260619 12:28:27.589574 24590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.581969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:37759 (local address 127.0.0.1:57538)
0619 12:28:27.583858 (+ 1889us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:27.583872 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:27.583968 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:27.584454 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:27.584458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:27.584476 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:27.584584 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.584612 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.585991 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.585994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.587234 (+ 1240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.587244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.588114 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.588117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.589216 (+ 1099us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.589245 (+ 29us) client_negotiation.cc:769] Sending connection context
0619 12:28:27.589337 (+ 92us) client_negotiation.cc:240] Negotiation successful
0619 12:28:27.589413 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":982,"mutex_wait_us":979,"thread_start_us":112,"threads_started":1}
I20260619 12:28:27.589715 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.583276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57538 (local address 127.6.231.194:37759)
0619 12:28:27.583611 (+ 335us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:27.583621 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:27.584069 (+ 448us) server_negotiation.cc:407] Connection header received
0619 12:28:27.584154 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:27.584158 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:27.584305 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:27.584464 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:27.584751 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.585838 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.587329 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.587982 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.588121 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.588274 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:27.589384 (+ 1110us) server_negotiation.cc:299] Negotiation successful
0619 12:28:27.589528 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":240,"thread_start_us":130,"threads_started":1}
I20260619 12:28:27.590783 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.582901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57656 (local address 127.6.231.195:45825)
0619 12:28:27.583340 (+ 439us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:27.583359 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:27.583382 (+ 23us) server_negotiation.cc:407] Connection header received
0619 12:28:27.583485 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:27.583490 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:27.583690 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:27.584097 (+ 407us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:27.584232 (+ 135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.585332 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.588232 (+ 2900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.588807 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.590301 (+ 1494us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.590470 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:27.590522 (+ 52us) server_negotiation.cc:299] Negotiation successful
0619 12:28:27.590621 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":145,"threads_started":1}
I20260619 12:28:27.590217 24587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.582703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:45825 (local address 127.0.0.1:57656)
0619 12:28:27.583167 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:27.583180 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:27.583298 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:27.583849 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:27.583852 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:27.583866 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:27.583954 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.583962 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.585550 (+ 1588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.585557 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.587123 (+ 1566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.587135 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.589917 (+ 2782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.589920 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.590004 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.590022 (+ 18us) client_negotiation.cc:769] Sending connection context
0619 12:28:27.590062 (+ 40us) client_negotiation.cc:240] Negotiation successful
0619 12:28:27.590115 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"mutex_wait_us":143,"thread_start_us":96,"threads_started":1}
I20260619 12:28:27.592105 24242 tablet_service.cc:1467] Processing CreateTablet for tablet 2077bc05987843b49d5c82b28aa3e907 (DEFAULT_TABLE table=test-table [id=cb25f9f921fc4d3cafb1e5fc79ebe3af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:27.593336 24242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2077bc05987843b49d5c82b28aa3e907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:27.593926 24510 tablet_service.cc:1467] Processing CreateTablet for tablet 2077bc05987843b49d5c82b28aa3e907 (DEFAULT_TABLE table=test-table [id=cb25f9f921fc4d3cafb1e5fc79ebe3af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:27.595098 24510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2077bc05987843b49d5c82b28aa3e907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:27.594805 24376 tablet_service.cc:1467] Processing CreateTablet for tablet 2077bc05987843b49d5c82b28aa3e907 (DEFAULT_TABLE table=test-table [id=cb25f9f921fc4d3cafb1e5fc79ebe3af]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:27.595822 24376 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2077bc05987843b49d5c82b28aa3e907. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:27.600479 24593 tablet_bootstrap.cc:492] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5: Bootstrap starting.
I20260619 12:28:27.602687 24594 tablet_bootstrap.cc:492] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5: Bootstrap starting.
I20260619 12:28:27.603089 24593 tablet_bootstrap.cc:654] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:27.603427 24595 tablet_bootstrap.cc:492] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a: Bootstrap starting.
I20260619 12:28:27.604233 24593 log.cc:824] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:27.605150 24594 tablet_bootstrap.cc:654] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:27.605894 24595 tablet_bootstrap.cc:654] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:27.606528 24594 log.cc:824] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:27.606702 24595 log.cc:824] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:27.606802 24593 tablet_bootstrap.cc:492] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5: No bootstrap required, opened a new log
I20260619 12:28:27.607087 24593 ts_tablet_manager.cc:1397] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.004s
I20260619 12:28:27.608738 24594 tablet_bootstrap.cc:492] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5: No bootstrap required, opened a new log
I20260619 12:28:27.608973 24594 ts_tablet_manager.cc:1397] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260619 12:28:27.609303 24595 tablet_bootstrap.cc:492] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a: No bootstrap required, opened a new log
I20260619 12:28:27.609550 24595 ts_tablet_manager.cc:1397] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.003s
I20260619 12:28:27.611892 24594 raft_consensus.cc:348] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c510349fb1a94b0280675820197661f5" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36111 } } peers { permanent_uuid: "4aef5ac0c3de46a589744579c562da2a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37759 } } peers { permanent_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45825 } }
I20260619 12:28:27.612221 24594 raft_consensus.cc:374] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:27.612270 24594 raft_consensus.cc:729] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2bd5af8dee81450297ebd6c9dafa28f5, State: Initialized, Role: FOLLOWER
I20260619 12:28:27.612576 24593 raft_consensus.cc:348] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c510349fb1a94b0280675820197661f5" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36111 } } peers { permanent_uuid: "4aef5ac0c3de46a589744579c562da2a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37759 } } peers { permanent_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45825 } }
I20260619 12:28:27.612748 24594 consensus_queue.cc:260] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c510349fb1a94b0280675820197661f5" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36111 } } peers { permanent_uuid: "4aef5ac0c3de46a589744579c562da2a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37759 } } peers { permanent_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45825 } }
I20260619 12:28:27.612865 24593 raft_consensus.cc:374] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:27.612920 24593 raft_consensus.cc:729] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c510349fb1a94b0280675820197661f5, State: Initialized, Role: FOLLOWER
I20260619 12:28:27.613466 24593 consensus_queue.cc:260] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c510349fb1a94b0280675820197661f5" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36111 } } peers { permanent_uuid: "4aef5ac0c3de46a589744579c562da2a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37759 } } peers { permanent_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45825 } }
I20260619 12:28:27.613870 24594 ts_tablet_manager.cc:1428] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260619 12:28:27.613978 24576 heartbeater.cc:503] Master 127.6.231.254:35917 was elected leader, sending a full tablet report...
I20260619 12:28:27.614282 24308 heartbeater.cc:503] Master 127.6.231.254:35917 was elected leader, sending a full tablet report...
I20260619 12:28:27.615103 24595 raft_consensus.cc:348] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c510349fb1a94b0280675820197661f5" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36111 } } peers { permanent_uuid: "4aef5ac0c3de46a589744579c562da2a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37759 } } peers { permanent_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45825 } }
I20260619 12:28:27.615477 24595 raft_consensus.cc:374] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:27.615581 24595 raft_consensus.cc:729] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4aef5ac0c3de46a589744579c562da2a, State: Initialized, Role: FOLLOWER
I20260619 12:28:27.615614 24593 ts_tablet_manager.cc:1428] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260619 12:28:27.616128 24595 consensus_queue.cc:260] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c510349fb1a94b0280675820197661f5" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36111 } } peers { permanent_uuid: "4aef5ac0c3de46a589744579c562da2a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37759 } } peers { permanent_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45825 } }
I20260619 12:28:27.616875 24442 heartbeater.cc:503] Master 127.6.231.254:35917 was elected leader, sending a full tablet report...
I20260619 12:28:27.617247 24595 ts_tablet_manager.cc:1428] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260619 12:28:27.628094 24599 raft_consensus.cc:482] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:28:27.628382 24599 raft_consensus.cc:504] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c510349fb1a94b0280675820197661f5" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36111 } } peers { permanent_uuid: "4aef5ac0c3de46a589744579c562da2a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37759 } } peers { permanent_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45825 } }
I20260619 12:28:27.630012 24599 leader_election.cc:288] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c510349fb1a94b0280675820197661f5 (127.6.231.193:36111), 4aef5ac0c3de46a589744579c562da2a (127.6.231.194:37759)
I20260619 12:28:27.634428 24592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.630453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:45217 (local address 127.6.231.194:37759)
0619 12:28:27.630605 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:27.630609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:27.630622 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:28:27.630669 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:27.630672 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:27.630714 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:27.630782 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:27.631131 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.631866 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.633220 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.633878 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.633975 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.634018 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:27.634259 (+ 241us) server_negotiation.cc:299] Negotiation successful
0619 12:28:27.634318 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 12:28:27.634480 24578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.630250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:37759 (local address 127.6.231.195:45217)
0619 12:28:27.630388 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:27.630407 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:27.630506 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:27.630862 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:27.630865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:27.630886 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:27.631009 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.631014 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.631985 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.631990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.633053 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.633063 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.633999 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.634003 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.634121 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.634149 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 12:28:27.634238 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 12:28:27.634324 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":1}
I20260619 12:28:27.635182 24602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.630274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:36111 (local address 127.6.231.195:49819)
0619 12:28:27.631022 (+ 748us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:27.631033 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:27.631149 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:27.631386 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:27.631389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:27.631408 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:27.631533 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.631539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.632561 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.632563 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.633554 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:27.633566 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.634709 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.634717 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:27.634832 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.634851 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 12:28:27.634954 (+ 103us) client_negotiation.cc:240] Negotiation successful
0619 12:28:27.635026 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":354,"thread_start_us":162,"threads_started":1}
I20260619 12:28:27.635177 24589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.630382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:49819 (local address 127.6.231.193:36111)
0619 12:28:27.630535 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:27.630540 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:27.631155 (+ 615us) server_negotiation.cc:407] Connection header received
0619 12:28:27.631232 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:27.631234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:27.631285 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:27.631348 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:27.631658 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.632415 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.633747 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.634438 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.634568 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.634621 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:27.634979 (+ 358us) server_negotiation.cc:299] Negotiation successful
0619 12:28:27.635058 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 12:28:27.635174 24396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2077bc05987843b49d5c82b28aa3e907" candidate_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aef5ac0c3de46a589744579c562da2a" is_pre_election: true
I20260619 12:28:27.635555 24396 raft_consensus.cc:2454] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bd5af8dee81450297ebd6c9dafa28f5 in term 0.
I20260619 12:28:27.635821 24262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2077bc05987843b49d5c82b28aa3e907" candidate_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c510349fb1a94b0280675820197661f5" is_pre_election: true
I20260619 12:28:27.636154 24262 raft_consensus.cc:2454] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2bd5af8dee81450297ebd6c9dafa28f5 in term 0.
I20260619 12:28:27.636155 24463 leader_election.cc:302] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2bd5af8dee81450297ebd6c9dafa28f5, 4aef5ac0c3de46a589744579c562da2a; no voters:
I20260619 12:28:27.636494 24599 raft_consensus.cc:2781] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:28:27.636605 24599 raft_consensus.cc:482] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:28:27.636655 24599 raft_consensus.cc:3037] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:27.637905 24599 raft_consensus.cc:504] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c510349fb1a94b0280675820197661f5" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36111 } } peers { permanent_uuid: "4aef5ac0c3de46a589744579c562da2a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37759 } } peers { permanent_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45825 } }
I20260619 12:28:27.638295 24599 leader_election.cc:288] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [CANDIDATE]: Term 1 election: Requested vote from peers c510349fb1a94b0280675820197661f5 (127.6.231.193:36111), 4aef5ac0c3de46a589744579c562da2a (127.6.231.194:37759)
I20260619 12:28:27.638784 24262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2077bc05987843b49d5c82b28aa3e907" candidate_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c510349fb1a94b0280675820197661f5"
I20260619 12:28:27.638989 24396 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2077bc05987843b49d5c82b28aa3e907" candidate_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4aef5ac0c3de46a589744579c562da2a"
I20260619 12:28:27.638965 24262 raft_consensus.cc:3037] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:27.639148 24396 raft_consensus.cc:3037] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:27.640585 24396 raft_consensus.cc:2454] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bd5af8dee81450297ebd6c9dafa28f5 in term 1.
I20260619 12:28:27.640585 24262 raft_consensus.cc:2454] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2bd5af8dee81450297ebd6c9dafa28f5 in term 1.
I20260619 12:28:27.641067 24466 leader_election.cc:302] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2bd5af8dee81450297ebd6c9dafa28f5, 4aef5ac0c3de46a589744579c562da2a, c510349fb1a94b0280675820197661f5; no voters:
I20260619 12:28:27.641353 24599 raft_consensus.cc:2781] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:27.641660 24599 raft_consensus.cc:686] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [term 1 LEADER]: Becoming Leader. State: Replica: 2bd5af8dee81450297ebd6c9dafa28f5, State: Running, Role: LEADER
I20260619 12:28:27.642040 24599 consensus_queue.cc:237] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c510349fb1a94b0280675820197661f5" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36111 } } peers { permanent_uuid: "4aef5ac0c3de46a589744579c562da2a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37759 } } peers { permanent_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45825 } }
I20260619 12:28:27.645960 24115 catalog_manager.cc:5495] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2bd5af8dee81450297ebd6c9dafa28f5 (127.6.231.195). New cstate: current_term: 1 leader_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c510349fb1a94b0280675820197661f5" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36111 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4aef5ac0c3de46a589744579c562da2a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2bd5af8dee81450297ebd6c9dafa28f5" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45825 } health_report { overall_health: HEALTHY } } }
I20260619 12:28:27.666749 24591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.662346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57672 (local address 127.6.231.195:45825)
0619 12:28:27.662524 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:27.662529 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:27.662545 (+ 16us) server_negotiation.cc:407] Connection header received
0619 12:28:27.662627 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:27.662630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:27.662707 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:27.662848 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:28:27.663143 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.663856 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.664995 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.665277 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.665463 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.665810 (+ 347us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:28:27.666317 (+ 507us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:28:27.666472 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:27.666517 (+ 45us) server_negotiation.cc:299] Negotiation successful
0619 12:28:27.666579 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 12:28:27.673386 24262 raft_consensus.cc:1261] T 2077bc05987843b49d5c82b28aa3e907 P c510349fb1a94b0280675820197661f5 [term 1 FOLLOWER]: Refusing update from remote peer 2bd5af8dee81450297ebd6c9dafa28f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:28:27.673656 24396 raft_consensus.cc:1261] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a [term 1 FOLLOWER]: Refusing update from remote peer 2bd5af8dee81450297ebd6c9dafa28f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:28:27.674301 24599 consensus_queue.cc:1035] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c510349fb1a94b0280675820197661f5" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36111 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:28:27.674515 24603 consensus_queue.cc:1035] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4aef5ac0c3de46a589744579c562da2a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 37759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:28:27.688496 24607 mvcc.cc:205] Tried to move back new op lower bound from 7298548153022042112 to 7298548152904810496. Current Snapshot: MvccSnapshot[applied={T|T < 7298548153022042112}]
I20260619 12:28:27.695854 24115 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:28:27.696081 24115 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:28:27.699131 24115 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd5f86e2041e4b699c90b58146e7035b: Sending DeleteTablet for 3 replicas of tablet 2077bc05987843b49d5c82b28aa3e907
I20260619 12:28:27.700146 24510 tablet_service.cc:1514] Processing DeleteTablet for tablet 2077bc05987843b49d5c82b28aa3e907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:27 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:57656
I20260619 12:28:27.700181 24376 tablet_service.cc:1514] Processing DeleteTablet for tablet 2077bc05987843b49d5c82b28aa3e907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:27 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:57538
I20260619 12:28:27.700214 24242 tablet_service.cc:1514] Processing DeleteTablet for tablet 2077bc05987843b49d5c82b28aa3e907 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:27 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:54570
I20260619 12:28:27.701323 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 24179
I20260619 12:28:27.705755 24616 tablet_replica.cc:330] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5: stopping tablet replica
I20260619 12:28:27.706131 24616 raft_consensus.cc:2229] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:28:27.706554 24616 raft_consensus.cc:2258] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:28:27.707022 24615 tablet_replica.cc:330] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a: stopping tablet replica
I20260619 12:28:27.707389 24615 raft_consensus.cc:2229] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:28:27.707732 24615 raft_consensus.cc:2258] T 2077bc05987843b49d5c82b28aa3e907 P 4aef5ac0c3de46a589744579c562da2a [term 1 FOLLOWER]: Raft consensus is shut down!
W20260619 12:28:27.712535 24104 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:36111 (error 108)
I20260619 12:28:27.713406 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 24313
I20260619 12:28:27.715998 24616 ts_tablet_manager.cc:1905] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:28:27.716460 24590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.716015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:36111 (local address 127.0.0.1:54578)
0619 12:28:27.716259 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:27.716344 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:36111: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
W20260619 12:28:27.716674 24104 catalog_manager.cc:4584] TS c510349fb1a94b0280675820197661f5 (127.6.231.193:36111): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2077bc05987843b49d5c82b28aa3e907: Network error: Client connection negotiation failed: client connection to 127.6.231.193:36111: connect: Connection refused (error 111)
I20260619 12:28:27.720788 24616 ts_tablet_manager.cc:1918] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:28:27.721014 24616 log.cc:1196] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/2077bc05987843b49d5c82b28aa3e907
I20260619 12:28:27.721535 24616 ts_tablet_manager.cc:1939] T 2077bc05987843b49d5c82b28aa3e907 P 2bd5af8dee81450297ebd6c9dafa28f5: Deleting consensus metadata
W20260619 12:28:27.721868 24590 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:28:27.721195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:37759 (local address 127.0.0.1:57542)
0619 12:28:27.721378 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:27.721394 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:27.721510 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:27.721747 (+ 237us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.194:37759: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":104}
I20260619 12:28:27.722452 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 24447
I20260619 12:28:27.728626 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 24085
2026-06-19T12:28:27Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2381 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-06-19T12:28:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:27Z Disabled control of system clock
I20260619 12:28:27.760389 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:36993
--webserver_interface=127.6.231.254
--webserver_port=0
--builtin_ntp_servers=127.6.231.212:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:36993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:28:27.875965 24622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:27.876276 24622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:27.876314 24622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:27.880457 24622 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:27.880533 24622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:27.880549 24622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:27.880566 24622 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:27.880582 24622 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:27.885578 24622 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:36993
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:36993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24622
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:27.887091 24622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:27.888865 24622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:27.898686 24627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:27.898698 24628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:27.898996 24622 server_base.cc:1034] running on GCE node
W20260619 12:28:27.898764 24630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:27.899693 24622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:27.900976 24622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:27.902161 24622 hybrid_clock.cc:648] HybridClock initialized: now 1781872107902133 us; error 44 us; skew 500 ppm
I20260619 12:28:27.904326 24622 webserver.cc:460] Webserver started at http://127.6.231.254:46513/ using document root <none> and password file <none>
I20260619 12:28:27.904982 24622 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:27.905049 24622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:27.905306 24622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:27.907513 24622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "f4bdf46428ba425e8f6ecb85db49bafc"
format_stamp: "Formatted at 2026-06-19 12:28:27 on dist-test-slave-kr9h"
server_key: "b40cc5129b60ffc2e019d88d4607462b"
server_key_iv: "caba7b8cad6d70793b93b86bee83f85c"
server_key_version: "encryptionkey@0"
I20260619 12:28:27.908115 24622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "f4bdf46428ba425e8f6ecb85db49bafc"
format_stamp: "Formatted at 2026-06-19 12:28:27 on dist-test-slave-kr9h"
server_key: "b40cc5129b60ffc2e019d88d4607462b"
server_key_iv: "caba7b8cad6d70793b93b86bee83f85c"
server_key_version: "encryptionkey@0"
I20260619 12:28:27.912119 24622 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 12:28:27.914817 24636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:27.916219 24622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 12:28:27.916343 24622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "f4bdf46428ba425e8f6ecb85db49bafc"
format_stamp: "Formatted at 2026-06-19 12:28:27 on dist-test-slave-kr9h"
server_key: "b40cc5129b60ffc2e019d88d4607462b"
server_key_iv: "caba7b8cad6d70793b93b86bee83f85c"
server_key_version: "encryptionkey@0"
I20260619 12:28:27.916504 24622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:27.929344 24622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:27.930238 24622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:27.930454 24622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:27.938220 24622 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:36993
I20260619 12:28:27.938236 24688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:36993 every 8 connection(s)
I20260619 12:28:27.939415 24622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:27.943632 24689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:27.947113 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 24622
I20260619 12:28:27.947273 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:27.947541 7071 external_mini_cluster.cc:1428] Setting key 9e26ef38b14ad5e8ca33f2a76c2d6c01
I20260619 12:28:27.950454 24689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc: Bootstrap starting.
I20260619 12:28:27.953578 24689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:27.954540 24689 log.cc:824] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:27.956938 24689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc: No bootstrap required, opened a new log
I20260619 12:28:27.959259 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:27.949130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34878 (local address 127.6.231.254:36993)
0619 12:28:27.949953 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:27.949975 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:27.950023 (+ 48us) server_negotiation.cc:407] Connection header received
0619 12:28:27.950973 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:27.951010 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:27.951560 (+ 550us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:27.952011 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:27.952504 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.954156 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.955142 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:27.955867 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:27.956046 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:27.956903 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:27.956910 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:27.956928 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:27.957533 (+ 605us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:27.957610 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:27.957637 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:27.957737 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:27.957916 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 12:28:27.958214 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":115,"threads_started":1}
I20260619 12:28:27.961946 24689 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4bdf46428ba425e8f6ecb85db49bafc" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36993 } }
I20260619 12:28:27.962244 24689 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:27.962304 24689 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4bdf46428ba425e8f6ecb85db49bafc, State: Initialized, Role: FOLLOWER
I20260619 12:28:27.962697 24689 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4bdf46428ba425e8f6ecb85db49bafc" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36993 } }
I20260619 12:28:27.962817 24689 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:27.962858 24689 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:27.962932 24689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:27.963979 24689 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4bdf46428ba425e8f6ecb85db49bafc" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36993 } }
I20260619 12:28:27.964341 24689 leader_election.cc:302] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4bdf46428ba425e8f6ecb85db49bafc; no voters:
I20260619 12:28:27.964743 24689 leader_election.cc:288] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:27.964876 24694 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:27.965163 24694 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [term 1 LEADER]: Becoming Leader. State: Replica: f4bdf46428ba425e8f6ecb85db49bafc, State: Running, Role: LEADER
I20260619 12:28:27.965538 24694 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4bdf46428ba425e8f6ecb85db49bafc" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36993 } }
I20260619 12:28:27.966194 24689 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:27.967319 24695 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4bdf46428ba425e8f6ecb85db49bafc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4bdf46428ba425e8f6ecb85db49bafc" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36993 } } }
I20260619 12:28:27.967485 24695 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:27.967314 24696 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4bdf46428ba425e8f6ecb85db49bafc. Latest consensus state: current_term: 1 leader_uuid: "f4bdf46428ba425e8f6ecb85db49bafc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4bdf46428ba425e8f6ecb85db49bafc" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36993 } } }
I20260619 12:28:27.967579 24696 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:27.967936 24703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:27.971240 24703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:27.977857 24703 catalog_manager.cc:1347] Generated new cluster ID: 8b1f741f0899493a8ef3802ef3dff01c
I20260619 12:28:27.977946 24703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:27.986125 24703 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:27.987197 24703 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:27.996547 24703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc: Generated new TSK 0
I20260619 12:28:27.997483 24703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:28.007047 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36993
--builtin_ntp_servers=127.6.231.212:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:28:28.123392 24713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:28.123661 24713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:28.123699 24713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:28.127774 24713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:28.127844 24713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:28.127902 24713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:28:28.132845 24713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:28.134032 24713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:28.135396 24713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:28.145519 24719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:28.145457 24718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:28.145569 24721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:28.145927 24713 server_base.cc:1034] running on GCE node
I20260619 12:28:28.146344 24713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:28.147011 24713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:28.148203 24713 hybrid_clock.cc:648] HybridClock initialized: now 1781872108148179 us; error 36 us; skew 500 ppm
I20260619 12:28:28.150346 24713 webserver.cc:460] Webserver started at http://127.6.231.193:39427/ using document root <none> and password file <none>
I20260619 12:28:28.151021 24713 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:28.151120 24713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:28.151371 24713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:28.153708 24713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "afeb71eb1d164d10a748689ff8d72fcb"
format_stamp: "Formatted at 2026-06-19 12:28:28 on dist-test-slave-kr9h"
server_key: "daf03a91f76260ec3a7abf7edb529329"
server_key_iv: "b27d61ff23ff237734b18b06a61d210d"
server_key_version: "encryptionkey@0"
I20260619 12:28:28.154295 24713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "afeb71eb1d164d10a748689ff8d72fcb"
format_stamp: "Formatted at 2026-06-19 12:28:28 on dist-test-slave-kr9h"
server_key: "daf03a91f76260ec3a7abf7edb529329"
server_key_iv: "b27d61ff23ff237734b18b06a61d210d"
server_key_version: "encryptionkey@0"
I20260619 12:28:28.158097 24713 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:28:28.160689 24727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:28.162017 24713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:28:28.162158 24713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "afeb71eb1d164d10a748689ff8d72fcb"
format_stamp: "Formatted at 2026-06-19 12:28:28 on dist-test-slave-kr9h"
server_key: "daf03a91f76260ec3a7abf7edb529329"
server_key_iv: "b27d61ff23ff237734b18b06a61d210d"
server_key_version: "encryptionkey@0"
I20260619 12:28:28.162285 24713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:28.171533 24713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:28.172246 24713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:28.172451 24713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:28.173044 24713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:28.174080 24713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:28.174153 24713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:28.174221 24713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:28.174283 24713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:28.183697 24713 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:37701
I20260619 12:28:28.183715 24840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:37701 every 8 connection(s)
I20260619 12:28:28.184670 24713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:28:28.194011 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 24713
I20260619 12:28:28.194160 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:28:28.194540 7071 external_mini_cluster.cc:1428] Setting key f0da10bbdd484ac610509554f178b903
I20260619 12:28:28.194666 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.186595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:34487 (local address 127.6.231.254:36993)
0619 12:28:28.186734 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:28.186808 (+ 74us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:28.187759 (+ 951us) server_negotiation.cc:407] Connection header received
0619 12:28:28.189288 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:28.189291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:28.189352 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:28.189467 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:28.190465 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.191088 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.192501 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.192720 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.192813 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.194045 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:28.194049 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:28.194051 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:28.194069 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:28.194094 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:28.194107 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:28.194192 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:28.194429 (+ 237us) server_negotiation.cc:299] Negotiation successful
0619 12:28:28.194498 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:28:28.195778 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.186958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36993 (local address 127.6.231.193:34487)
0619 12:28:28.187622 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:28.187664 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:28.189008 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:28.189660 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:28.189667 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:28.189806 (+ 139us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:28.190307 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.190319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.191207 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.191210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.192381 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.192390 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.192871 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.192874 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.193012 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.193662 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:28:28.193664 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:28.193705 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:28:28.193779 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:28.194233 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:28.194250 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:28.194257 (+ 7us) client_negotiation.cc:769] Sending connection context
0619 12:28:28.194381 (+ 124us) client_negotiation.cc:240] Negotiation successful
0619 12:28:28.194642 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":118,"threads_started":1}
I20260619 12:28:28.196877 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36993
--builtin_ntp_servers=127.6.231.212:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:28:28.197104 24841 heartbeater.cc:347] Connected to a master server at 127.6.231.254:36993
I20260619 12:28:28.197358 24841 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:28.197918 24841 heartbeater.cc:511] Master 127.6.231.254:36993 requested a full tablet report, sending...
I20260619 12:28:28.199517 24653 ts_manager.cc:194] Registered new tserver with Master: afeb71eb1d164d10a748689ff8d72fcb (127.6.231.193:37701)
I20260619 12:28:28.200665 24653 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.231.193:34487
W20260619 12:28:28.313607 24844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:28.314127 24844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:28.314172 24844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:28.318480 24844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:28.318549 24844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:28.318645 24844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:28:28.324033 24844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:28.325227 24844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:28.326742 24844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:28.336905 24849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:28.336925 24850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:28.336874 24852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:28.337239 24844 server_base.cc:1034] running on GCE node
I20260619 12:28:28.337841 24844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:28.338572 24844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:28.339846 24844 hybrid_clock.cc:648] HybridClock initialized: now 1781872108339807 us; error 66 us; skew 500 ppm
I20260619 12:28:28.342036 24844 webserver.cc:460] Webserver started at http://127.6.231.194:35233/ using document root <none> and password file <none>
I20260619 12:28:28.342654 24844 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:28.342759 24844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:28.342999 24844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:28.345117 24844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52"
format_stamp: "Formatted at 2026-06-19 12:28:28 on dist-test-slave-kr9h"
server_key: "a132ad83a57c4923048efef1b0cda98b"
server_key_iv: "21ef16f6a0f77c8dc617866a008a9a06"
server_key_version: "encryptionkey@0"
I20260619 12:28:28.345710 24844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52"
format_stamp: "Formatted at 2026-06-19 12:28:28 on dist-test-slave-kr9h"
server_key: "a132ad83a57c4923048efef1b0cda98b"
server_key_iv: "21ef16f6a0f77c8dc617866a008a9a06"
server_key_version: "encryptionkey@0"
I20260619 12:28:28.349987 24844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 12:28:28.352761 24858 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:28.354373 24844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:28.354539 24844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52"
format_stamp: "Formatted at 2026-06-19 12:28:28 on dist-test-slave-kr9h"
server_key: "a132ad83a57c4923048efef1b0cda98b"
server_key_iv: "21ef16f6a0f77c8dc617866a008a9a06"
server_key_version: "encryptionkey@0"
I20260619 12:28:28.354677 24844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:28.374954 24844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:28.375735 24844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:28.375960 24844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:28.376585 24844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:28.377663 24844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:28.377732 24844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:28.377830 24844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:28.377867 24844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:28.387013 24844 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:41403
I20260619 12:28:28.387030 24971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:41403 every 8 connection(s)
I20260619 12:28:28.387995 24844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:28:28.393972 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 24844
I20260619 12:28:28.394141 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:28:28.394445 7071 external_mini_cluster.cc:1428] Setting key 8b1887a98f5663092ea4d4db9ae783a1
I20260619 12:28:28.396706 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36993
--builtin_ntp_servers=127.6.231.212:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:28:28.398242 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.389911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:34319 (local address 127.6.231.254:36993)
0619 12:28:28.390085 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:28.390091 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:28.391279 (+ 1188us) server_negotiation.cc:407] Connection header received
0619 12:28:28.392781 (+ 1502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:28.392784 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:28.392836 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:28.392917 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:28.394117 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.394748 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.395966 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.396219 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.396308 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.397774 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:28.397777 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:28.397779 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:28.397792 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:28.397813 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:28.397822 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:28.397878 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:28.398073 (+ 195us) server_negotiation.cc:299] Negotiation successful
0619 12:28:28.398128 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 12:28:28.399315 24974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.390411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36993 (local address 127.6.231.194:34319)
0619 12:28:28.391110 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:28.391152 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:28.392568 (+ 1416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:28.393091 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:28.393100 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:28.393217 (+ 117us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:28.393923 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.393944 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.394909 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.394912 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.395845 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.395854 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.396392 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.396399 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.396640 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.397384 (+ 744us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:28:28.397389 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:28.397492 (+ 103us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:28:28.397593 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:28.397905 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:28.397917 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:28.397926 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 12:28:28.398064 (+ 138us) client_negotiation.cc:240] Negotiation successful
0619 12:28:28.398422 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":119,"threads_started":1}
I20260619 12:28:28.400655 24972 heartbeater.cc:347] Connected to a master server at 127.6.231.254:36993
I20260619 12:28:28.400959 24972 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:28.401590 24972 heartbeater.cc:511] Master 127.6.231.254:36993 requested a full tablet report, sending...
I20260619 12:28:28.402763 24653 ts_manager.cc:194] Registered new tserver with Master: 95c9aa2f3f4c4306a56c6451aa1a9b52 (127.6.231.194:41403)
I20260619 12:28:28.403687 24653 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.231.194:34319
W20260619 12:28:28.513034 24975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:28.513335 24975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:28.513375 24975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:28.517560 24975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:28.517628 24975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:28.517686 24975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:28:28.522662 24975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:28.523841 24975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:28.525329 24975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:28.535261 24983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:28.535282 24980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:28.535343 24975 server_base.cc:1034] running on GCE node
W20260619 12:28:28.535315 24981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:28.535945 24975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:28.536684 24975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:28.537889 24975 hybrid_clock.cc:648] HybridClock initialized: now 1781872108537868 us; error 43 us; skew 500 ppm
I20260619 12:28:28.540052 24975 webserver.cc:460] Webserver started at http://127.6.231.195:35979/ using document root <none> and password file <none>
I20260619 12:28:28.540699 24975 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:28.540807 24975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:28.541047 24975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:28.543408 24975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "ba7261acf25e4c81893ca4892f4cc706"
format_stamp: "Formatted at 2026-06-19 12:28:28 on dist-test-slave-kr9h"
server_key: "c22d4087c987fa69e773ce64b4a4056d"
server_key_iv: "743c23f0437cd078c56486c91416e5f4"
server_key_version: "encryptionkey@0"
I20260619 12:28:28.544020 24975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "ba7261acf25e4c81893ca4892f4cc706"
format_stamp: "Formatted at 2026-06-19 12:28:28 on dist-test-slave-kr9h"
server_key: "c22d4087c987fa69e773ce64b4a4056d"
server_key_iv: "743c23f0437cd078c56486c91416e5f4"
server_key_version: "encryptionkey@0"
I20260619 12:28:28.548189 24975 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 12:28:28.551100 24989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:28.552480 24975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260619 12:28:28.552630 24975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "ba7261acf25e4c81893ca4892f4cc706"
format_stamp: "Formatted at 2026-06-19 12:28:28 on dist-test-slave-kr9h"
server_key: "c22d4087c987fa69e773ce64b4a4056d"
server_key_iv: "743c23f0437cd078c56486c91416e5f4"
server_key_version: "encryptionkey@0"
I20260619 12:28:28.552757 24975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:28.571055 24975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:28.571883 24975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:28.572117 24975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:28.572811 24975 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:28.573987 24975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:28.574067 24975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:28.574146 24975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:28.574203 24975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:28.583956 24975 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:45217
I20260619 12:28:28.583982 25102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:45217 every 8 connection(s)
I20260619 12:28:28.585001 24975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:28:28.594372 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 24975
I20260619 12:28:28.594519 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:28:28.594972 7071 external_mini_cluster.cc:1428] Setting key e8076aade3add043cd59e44e9e8e2f47
I20260619 12:28:28.595525 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.586945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:48579 (local address 127.6.231.254:36993)
0619 12:28:28.587097 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:28.587101 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:28.588278 (+ 1177us) server_negotiation.cc:407] Connection header received
0619 12:28:28.589871 (+ 1593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:28.589875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:28.589938 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:28.590063 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:28.591078 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.591711 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.593128 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.593390 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.593561 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.594943 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:28.594948 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:28.594951 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:28.594968 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:28.594996 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:28.595010 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:28.595104 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:28.595302 (+ 198us) server_negotiation.cc:299] Negotiation successful
0619 12:28:28.595373 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 12:28:28.596637 25105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.587324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36993 (local address 127.6.231.195:48579)
0619 12:28:28.588066 (+ 742us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:28.588112 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:28.589611 (+ 1499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:28.590264 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:28.590273 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:28.590377 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:28.590910 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.590924 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.591857 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.591862 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.592989 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.592999 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.593606 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.593611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.593805 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.594579 (+ 774us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:28:28.594581 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:28.594663 (+ 82us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:28:28.594754 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:28.595117 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:28.595128 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:28.595137 (+ 9us) client_negotiation.cc:769] Sending connection context
0619 12:28:28.595258 (+ 121us) client_negotiation.cc:240] Negotiation successful
0619 12:28:28.595560 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":110,"threads_started":1}
I20260619 12:28:28.597887 25103 heartbeater.cc:347] Connected to a master server at 127.6.231.254:36993
I20260619 12:28:28.598161 25103 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:28.598731 25103 heartbeater.cc:511] Master 127.6.231.254:36993 requested a full tablet report, sending...
I20260619 12:28:28.599965 24653 ts_manager.cc:194] Registered new tserver with Master: ba7261acf25e4c81893ca4892f4cc706 (127.6.231.195:45217)
I20260619 12:28:28.600570 24653 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.231.195:48579
I20260619 12:28:28.609601 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:28:28.615773 24692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.611764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34886 (local address 127.6.231.254:36993)
0619 12:28:28.611962 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:28.611967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:28.612100 (+ 133us) server_negotiation.cc:407] Connection header received
0619 12:28:28.612260 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:28.612264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:28.612321 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:28.612411 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:28.612699 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.613230 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.614112 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.614335 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.614444 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.615294 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:28.615297 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:28.615300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:28.615312 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:28.615331 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:28.615339 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:28.615426 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:28.615556 (+ 130us) server_negotiation.cc:299] Negotiation successful
0619 12:28:28.615617 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260619 12:28:28.618988 24653 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:34886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:28:28.622486 24653 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:28:28.638491 25114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.632755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:41403 (local address 127.0.0.1:44476)
0619 12:28:28.633200 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:28.633213 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:28.633279 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:28.634006 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:28.634015 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:28.634045 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:28.634311 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.634320 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.635414 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.635419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.636753 (+ 1334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.636763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.637930 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.637933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.638041 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.638079 (+ 38us) client_negotiation.cc:769] Sending connection context
0619 12:28:28.638169 (+ 90us) client_negotiation.cc:240] Negotiation successful
0619 12:28:28.638332 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":88,"thread_start_us":134,"threads_started":1}
I20260619 12:28:28.639146 25116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.632181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36930 (local address 127.6.231.195:45217)
0619 12:28:28.632701 (+ 520us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:28.632714 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:28.633144 (+ 430us) server_negotiation.cc:407] Connection header received
0619 12:28:28.633499 (+ 355us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:28.633506 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:28.633735 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:28.633867 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:28.634434 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.635682 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.637005 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.637783 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.637905 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.638109 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:28.638841 (+ 732us) server_negotiation.cc:299] Negotiation successful
0619 12:28:28.638957 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":163,"threads_started":1}
I20260619 12:28:28.639267 25117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.632515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43770 (local address 127.6.231.193:37701)
0619 12:28:28.632898 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:28.632910 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:28.633640 (+ 730us) server_negotiation.cc:407] Connection header received
0619 12:28:28.633773 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:28.633778 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:28.633962 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:28.634109 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:28.634524 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.635665 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.637890 (+ 2225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.638432 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.638563 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.638765 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:28.639020 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 12:28:28.639120 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":84,"threads_started":1}
I20260619 12:28:28.639192 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.633153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44476 (local address 127.6.231.194:41403)
0619 12:28:28.633586 (+ 433us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:28.633598 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:28.633622 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:28:28.633700 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:28.633705 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:28.633855 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:28.633984 (+ 129us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:28.634491 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.635287 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.636898 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.637783 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.638297 (+ 514us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.638489 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:28.638548 (+ 59us) server_negotiation.cc:299] Negotiation successful
0619 12:28:28.639036 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":97,"threads_started":1}
I20260619 12:28:28.639216 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.632001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:37701 (local address 127.0.0.1:43770)
0619 12:28:28.633527 (+ 1526us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:28.633539 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:28.633652 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:28.634142 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:28.634145 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:28.634160 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:28.634359 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.634368 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.636991 (+ 2623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.636993 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.637767 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.637775 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.638563 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.638566 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.638852 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.638874 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 12:28:28.638986 (+ 112us) client_negotiation.cc:240] Negotiation successful
0619 12:28:28.639058 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1032,"mutex_wait_us":92,"thread_start_us":78,"threads_started":1}
I20260619 12:28:28.639691 25115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.631979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:45217 (local address 127.0.0.1:36930)
0619 12:28:28.632995 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:28.633049 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:28.633279 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:28.634066 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:28.634069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:28.634088 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:28.634306 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.634313 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.635801 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.635804 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.636889 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.636896 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.638169 (+ 1273us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.638172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.638680 (+ 508us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.638706 (+ 26us) client_negotiation.cc:769] Sending connection context
0619 12:28:28.639473 (+ 767us) client_negotiation.cc:240] Negotiation successful
0619 12:28:28.639566 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":26,"spinlock_wait_cycles":10496,"thread_start_us":134,"threads_started":1}
I20260619 12:28:28.642360 24906 tablet_service.cc:1467] Processing CreateTablet for tablet 14ca618bf29f407d9a8a060546a77781 (DEFAULT_TABLE table=test-table [id=6598b619ae444955acbbc3e3972b4a23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:28.642402 25037 tablet_service.cc:1467] Processing CreateTablet for tablet 14ca618bf29f407d9a8a060546a77781 (DEFAULT_TABLE table=test-table [id=6598b619ae444955acbbc3e3972b4a23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:28.642402 24775 tablet_service.cc:1467] Processing CreateTablet for tablet 14ca618bf29f407d9a8a060546a77781 (DEFAULT_TABLE table=test-table [id=6598b619ae444955acbbc3e3972b4a23]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:28.643378 24906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14ca618bf29f407d9a8a060546a77781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:28.643392 25037 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14ca618bf29f407d9a8a060546a77781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:28.643393 24775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 14ca618bf29f407d9a8a060546a77781. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:28.650094 25120 tablet_bootstrap.cc:492] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52: Bootstrap starting.
I20260619 12:28:28.651126 25121 tablet_bootstrap.cc:492] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706: Bootstrap starting.
I20260619 12:28:28.652581 25120 tablet_bootstrap.cc:654] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:28.652947 25121 tablet_bootstrap.cc:654] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:28.653375 25120 log.cc:824] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:28.653709 25121 log.cc:824] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:28.653844 25122 tablet_bootstrap.cc:492] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb: Bootstrap starting.
I20260619 12:28:28.655625 25120 tablet_bootstrap.cc:492] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52: No bootstrap required, opened a new log
I20260619 12:28:28.655697 25121 tablet_bootstrap.cc:492] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706: No bootstrap required, opened a new log
I20260619 12:28:28.655836 25120 ts_tablet_manager.cc:1397] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260619 12:28:28.655884 25121 ts_tablet_manager.cc:1397] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 12:28:28.655947 25122 tablet_bootstrap.cc:654] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:28.656834 25122 log.cc:824] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:28.659104 25122 tablet_bootstrap.cc:492] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb: No bootstrap required, opened a new log
I20260619 12:28:28.659322 25122 ts_tablet_manager.cc:1397] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 12:28:28.659931 25120 raft_consensus.cc:348] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7261acf25e4c81893ca4892f4cc706" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45217 } } peers { permanent_uuid: "afeb71eb1d164d10a748689ff8d72fcb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 37701 } } peers { permanent_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41403 } }
I20260619 12:28:28.660243 25120 raft_consensus.cc:374] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:28.660300 25120 raft_consensus.cc:729] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 95c9aa2f3f4c4306a56c6451aa1a9b52, State: Initialized, Role: FOLLOWER
I20260619 12:28:28.660265 25121 raft_consensus.cc:348] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7261acf25e4c81893ca4892f4cc706" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45217 } } peers { permanent_uuid: "afeb71eb1d164d10a748689ff8d72fcb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 37701 } } peers { permanent_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41403 } }
I20260619 12:28:28.660518 25121 raft_consensus.cc:374] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:28.660578 25121 raft_consensus.cc:729] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba7261acf25e4c81893ca4892f4cc706, State: Initialized, Role: FOLLOWER
I20260619 12:28:28.660827 25120 consensus_queue.cc:260] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7261acf25e4c81893ca4892f4cc706" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45217 } } peers { permanent_uuid: "afeb71eb1d164d10a748689ff8d72fcb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 37701 } } peers { permanent_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41403 } }
I20260619 12:28:28.660998 25121 consensus_queue.cc:260] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7261acf25e4c81893ca4892f4cc706" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45217 } } peers { permanent_uuid: "afeb71eb1d164d10a748689ff8d72fcb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 37701 } } peers { permanent_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41403 } }
I20260619 12:28:28.661927 25120 ts_tablet_manager.cc:1428] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52: Time spent starting tablet: real 0.006s user 0.004s sys 0.001s
I20260619 12:28:28.661962 25121 ts_tablet_manager.cc:1428] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 12:28:28.662326 24972 heartbeater.cc:503] Master 127.6.231.254:36993 was elected leader, sending a full tablet report...
I20260619 12:28:28.663383 25103 heartbeater.cc:503] Master 127.6.231.254:36993 was elected leader, sending a full tablet report...
I20260619 12:28:28.663626 25122 raft_consensus.cc:348] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7261acf25e4c81893ca4892f4cc706" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45217 } } peers { permanent_uuid: "afeb71eb1d164d10a748689ff8d72fcb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 37701 } } peers { permanent_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41403 } }
I20260619 12:28:28.663887 25122 raft_consensus.cc:374] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:28.663935 25122 raft_consensus.cc:729] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afeb71eb1d164d10a748689ff8d72fcb, State: Initialized, Role: FOLLOWER
I20260619 12:28:28.664376 25122 consensus_queue.cc:260] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7261acf25e4c81893ca4892f4cc706" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45217 } } peers { permanent_uuid: "afeb71eb1d164d10a748689ff8d72fcb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 37701 } } peers { permanent_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41403 } }
I20260619 12:28:28.664820 25126 raft_consensus.cc:482] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:28:28.664944 25126 raft_consensus.cc:504] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7261acf25e4c81893ca4892f4cc706" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45217 } } peers { permanent_uuid: "afeb71eb1d164d10a748689ff8d72fcb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 37701 } } peers { permanent_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41403 } }
I20260619 12:28:28.665032 24841 heartbeater.cc:503] Master 127.6.231.254:36993 was elected leader, sending a full tablet report...
I20260619 12:28:28.665406 25122 ts_tablet_manager.cc:1428] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 12:28:28.666237 25126 leader_election.cc:288] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ba7261acf25e4c81893ca4892f4cc706 (127.6.231.195:45217), afeb71eb1d164d10a748689ff8d72fcb (127.6.231.193:37701)
I20260619 12:28:28.670940 24974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.666320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:45217 (local address 127.6.231.194:59997)
0619 12:28:28.666428 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:28.666444 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:28.666535 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:28.667228 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:28.667230 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:28.667265 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:28.667381 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.667390 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.668483 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.668489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.669802 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.669811 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.670458 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.670462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.670588 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.670612 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 12:28:28.670714 (+ 102us) client_negotiation.cc:240] Negotiation successful
0619 12:28:28.670789 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27,"mutex_wait_us":3,"spinlock_wait_cycles":22912}
I20260619 12:28:28.670935 25116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.666866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:59997 (local address 127.6.231.195:45217)
0619 12:28:28.666999 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:28.667006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:28.667023 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:28:28.667072 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:28.667075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:28.667123 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:28.667202 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:28.667501 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.668195 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.669923 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.670336 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.670465 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.670517 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:28.670738 (+ 221us) server_negotiation.cc:299] Negotiation successful
0619 12:28:28.670811 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260619 12:28:28.671576 25057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14ca618bf29f407d9a8a060546a77781" candidate_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7261acf25e4c81893ca4892f4cc706" is_pre_election: true
I20260619 12:28:28.671880 25057 raft_consensus.cc:2454] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95c9aa2f3f4c4306a56c6451aa1a9b52 in term 0.
I20260619 12:28:28.672334 25130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.666454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:37701 (local address 127.6.231.194:57945)
0619 12:28:28.666769 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:28.666784 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:28.666872 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:28.668514 (+ 1642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:28.668517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:28.668535 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:28.668646 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.668692 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.669717 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.669721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.670988 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:28.671000 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.671891 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.671894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:28.671964 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.672006 (+ 42us) client_negotiation.cc:769] Sending connection context
0619 12:28:28.672076 (+ 70us) client_negotiation.cc:240] Negotiation successful
0619 12:28:28.672162 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":23,"thread_start_us":129,"threads_started":1}
I20260619 12:28:28.672330 25117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.667685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:57945 (local address 127.6.231.193:37701)
0619 12:28:28.667894 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:28.667899 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:28.667913 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:28:28.667956 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:28.667959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:28.668001 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:28.668084 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:28.668816 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.669606 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.671134 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.671648 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.671751 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.671798 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:28.672108 (+ 310us) server_negotiation.cc:299] Negotiation successful
0619 12:28:28.672194 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135}
I20260619 12:28:28.672416 24860 leader_election.cc:302] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95c9aa2f3f4c4306a56c6451aa1a9b52, ba7261acf25e4c81893ca4892f4cc706; no voters:
I20260619 12:28:28.672679 25126 raft_consensus.cc:2781] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:28:28.672786 25126 raft_consensus.cc:482] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:28:28.672847 25126 raft_consensus.cc:3037] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:28.672910 24795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14ca618bf29f407d9a8a060546a77781" candidate_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afeb71eb1d164d10a748689ff8d72fcb" is_pre_election: true
I20260619 12:28:28.673194 24795 raft_consensus.cc:2454] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 95c9aa2f3f4c4306a56c6451aa1a9b52 in term 0.
I20260619 12:28:28.674131 25126 raft_consensus.cc:504] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7261acf25e4c81893ca4892f4cc706" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45217 } } peers { permanent_uuid: "afeb71eb1d164d10a748689ff8d72fcb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 37701 } } peers { permanent_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41403 } }
I20260619 12:28:28.674479 25126 leader_election.cc:288] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [CANDIDATE]: Term 1 election: Requested vote from peers ba7261acf25e4c81893ca4892f4cc706 (127.6.231.195:45217), afeb71eb1d164d10a748689ff8d72fcb (127.6.231.193:37701)
I20260619 12:28:28.674836 25057 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14ca618bf29f407d9a8a060546a77781" candidate_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ba7261acf25e4c81893ca4892f4cc706"
I20260619 12:28:28.674985 25057 raft_consensus.cc:3037] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:28.675001 24795 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "14ca618bf29f407d9a8a060546a77781" candidate_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afeb71eb1d164d10a748689ff8d72fcb"
I20260619 12:28:28.675110 24795 raft_consensus.cc:3037] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:28.676333 24795 raft_consensus.cc:2454] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95c9aa2f3f4c4306a56c6451aa1a9b52 in term 1.
I20260619 12:28:28.676337 25057 raft_consensus.cc:2454] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 95c9aa2f3f4c4306a56c6451aa1a9b52 in term 1.
I20260619 12:28:28.676630 24860 leader_election.cc:302] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 95c9aa2f3f4c4306a56c6451aa1a9b52, ba7261acf25e4c81893ca4892f4cc706; no voters:
I20260619 12:28:28.676791 25126 raft_consensus.cc:2781] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:28.676949 25126 raft_consensus.cc:686] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [term 1 LEADER]: Becoming Leader. State: Replica: 95c9aa2f3f4c4306a56c6451aa1a9b52, State: Running, Role: LEADER
I20260619 12:28:28.677271 25126 consensus_queue.cc:237] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7261acf25e4c81893ca4892f4cc706" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45217 } } peers { permanent_uuid: "afeb71eb1d164d10a748689ff8d72fcb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 37701 } } peers { permanent_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41403 } }
I20260619 12:28:28.680539 24652 catalog_manager.cc:5495] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 reported cstate change: term changed from 0 to 1, leader changed from <none> to 95c9aa2f3f4c4306a56c6451aa1a9b52 (127.6.231.194). New cstate: current_term: 1 leader_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ba7261acf25e4c81893ca4892f4cc706" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45217 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afeb71eb1d164d10a748689ff8d72fcb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 37701 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "95c9aa2f3f4c4306a56c6451aa1a9b52" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41403 } health_report { overall_health: HEALTHY } } }
W20260619 12:28:28.685887 24842 tablet.cc:2369] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:28:28.703377 25119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.699916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44478 (local address 127.6.231.194:41403)
0619 12:28:28.700051 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:28.700054 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:28.700068 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:28:28.700147 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:28.700150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:28.700219 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:28.700330 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:28:28.700573 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.701112 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.701997 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:28.702245 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:28.702336 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:28.702609 (+ 273us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:28:28.703056 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:28:28.703158 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:28.703206 (+ 48us) server_negotiation.cc:299] Negotiation successful
0619 12:28:28.703255 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:28:28.709532 25057 raft_consensus.cc:1261] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706 [term 1 FOLLOWER]: Refusing update from remote peer 95c9aa2f3f4c4306a56c6451aa1a9b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:28:28.709537 24795 raft_consensus.cc:1261] T 14ca618bf29f407d9a8a060546a77781 P afeb71eb1d164d10a748689ff8d72fcb [term 1 FOLLOWER]: Refusing update from remote peer 95c9aa2f3f4c4306a56c6451aa1a9b52: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:28:28.710312 25128 consensus_queue.cc:1035] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ba7261acf25e4c81893ca4892f4cc706" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 45217 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:28:28.710513 25126 consensus_queue.cc:1035] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [LEADER]: Connected to new peer: Peer: permanent_uuid: "afeb71eb1d164d10a748689ff8d72fcb" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 37701 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:28:28.718833 25135 mvcc.cc:205] Tried to move back new op lower bound from 7298548157266567168 to 7298548157145137152. Current Snapshot: MvccSnapshot[applied={T|T < 7298548157266567168}]
I20260619 12:28:28.728629 24652 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:34886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:28:28.728839 24652 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:34886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:28:28.731526 24652 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f4bdf46428ba425e8f6ecb85db49bafc: Sending DeleteTablet for 3 replicas of tablet 14ca618bf29f407d9a8a060546a77781
I20260619 12:28:28.732395 25037 tablet_service.cc:1514] Processing DeleteTablet for tablet 14ca618bf29f407d9a8a060546a77781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:28 UTC) from {username='slave'} at 127.0.0.1:36930
I20260619 12:28:28.732503 24906 tablet_service.cc:1514] Processing DeleteTablet for tablet 14ca618bf29f407d9a8a060546a77781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:28 UTC) from {username='slave'} at 127.0.0.1:44476
I20260619 12:28:28.733042 24775 tablet_service.cc:1514] Processing DeleteTablet for tablet 14ca618bf29f407d9a8a060546a77781 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:28 UTC) from {username='slave'} at 127.0.0.1:43770
I20260619 12:28:28.733239 25143 tablet_replica.cc:330] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52: stopping tablet replica
I20260619 12:28:28.733577 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 24713
I20260619 12:28:28.733640 25143 raft_consensus.cc:2229] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:28:28.736145 25143 raft_consensus.cc:2258] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:28:28.736617 25142 tablet_replica.cc:330] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706: stopping tablet replica
I20260619 12:28:28.736982 25142 raft_consensus.cc:2229] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:28:28.737332 25142 raft_consensus.cc:2258] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:28:28.738291 25143 ts_tablet_manager.cc:1905] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:28:28.738801 25142 ts_tablet_manager.cc:1905] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260619 12:28:28.741143 24637 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:37701 (error 108)
I20260619 12:28:28.741475 25142 ts_tablet_manager.cc:1918] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:28:28.741587 25142 log.cc:1196] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/14ca618bf29f407d9a8a060546a77781
I20260619 12:28:28.741719 25143 ts_tablet_manager.cc:1918] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:28:28.741905 25143 log.cc:1196] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/14ca618bf29f407d9a8a060546a77781
I20260619 12:28:28.741926 25142 ts_tablet_manager.cc:1939] T 14ca618bf29f407d9a8a060546a77781 P ba7261acf25e4c81893ca4892f4cc706: Deleting consensus metadata
I20260619 12:28:28.742272 25143 ts_tablet_manager.cc:1939] T 14ca618bf29f407d9a8a060546a77781 P 95c9aa2f3f4c4306a56c6451aa1a9b52: Deleting consensus metadata
I20260619 12:28:28.742578 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 24844
I20260619 12:28:28.742641 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.742323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:37701 (local address 127.0.0.1:43772)
0619 12:28:28.742471 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:28.742551 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:37701: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 12:28:28.743777 24638 catalog_manager.cc:4841] TS ba7261acf25e4c81893ca4892f4cc706 (127.6.231.195:45217): tablet 14ca618bf29f407d9a8a060546a77781 (table test-table [id=6598b619ae444955acbbc3e3972b4a23]) successfully deleted
W20260619 12:28:28.744287 24637 catalog_manager.cc:4584] TS afeb71eb1d164d10a748689ff8d72fcb (127.6.231.193:37701): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 14ca618bf29f407d9a8a060546a77781: Network error: Client connection negotiation failed: client connection to 127.6.231.193:37701: connect: Connection refused (error 111)
I20260619 12:28:28.748721 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 24975
I20260619 12:28:28.749511 25118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:28.748890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:41403 (local address 127.0.0.1:44494)
0619 12:28:28.749323 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:28.749385 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.194:41403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":334}
I20260619 12:28:28.754284 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 24622
2026-06-19T12:28:28Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1024 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:28 dist-test-slave-kr9h krb5kdc[25148](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:28 dist-test-slave-kr9h krb5kdc[25148](info): set up 2 sockets
Jun 19 12:28:28 dist-test-slave-kr9h krb5kdc[25148](info): commencing operation
krb5kdc: starting...
W20260619 12:28:29.793592 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:29 dist-test-slave-kr9h krb5kdc[25148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:28:29.922964 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:36243
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:46589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:36243
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:30.036936 25161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:30.037247 25161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:30.037290 25161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:30.041453 25161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:30.041531 25161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:30.041553 25161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:30.041571 25161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:30.041587 25161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:30.047159 25161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:46589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:36243
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:36243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:30.048368 25161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:30.049924 25161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:30.058786 25169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:30.059000 25166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:30.058800 25167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:30.059072 25161 server_base.cc:1034] running on GCE node
I20260619 12:28:30.059907 25161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:30.061195 25161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:30.062402 25161 hybrid_clock.cc:648] HybridClock initialized: now 1781872110062386 us; error 76 us; skew 500 ppm
Jun 19 12:28:30 dist-test-slave-kr9h krb5kdc[25148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872110, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:30.070008 25161 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:28:30.071118 25161 webserver.cc:460] Webserver started at http://127.6.231.254:41599/ using document root <none> and password file <none>
I20260619 12:28:30.071764 25161 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:30.071873 25161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:30.072134 25161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:30.074625 25161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "c74358f468694f6c982ddf468a8e5d26"
format_stamp: "Formatted at 2026-06-19 12:28:30 on dist-test-slave-kr9h"
server_key: "24f0ee7842d88ac456f94af7b0b7cba0"
server_key_iv: "b1312f8e20f0bbde51a035ba606bda92"
server_key_version: "encryptionkey@0"
I20260619 12:28:30.075279 25161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "c74358f468694f6c982ddf468a8e5d26"
format_stamp: "Formatted at 2026-06-19 12:28:30 on dist-test-slave-kr9h"
server_key: "24f0ee7842d88ac456f94af7b0b7cba0"
server_key_iv: "b1312f8e20f0bbde51a035ba606bda92"
server_key_version: "encryptionkey@0"
I20260619 12:28:30.079725 25161 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:28:30.082437 25176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:30.083801 25161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:30.083979 25161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "c74358f468694f6c982ddf468a8e5d26"
format_stamp: "Formatted at 2026-06-19 12:28:30 on dist-test-slave-kr9h"
server_key: "24f0ee7842d88ac456f94af7b0b7cba0"
server_key_iv: "b1312f8e20f0bbde51a035ba606bda92"
server_key_version: "encryptionkey@0"
I20260619 12:28:30.084118 25161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:30.095798 25161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:30.099241 25161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:30.099475 25161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:30.107111 25161 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:36243
I20260619 12:28:30.107136 25228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:36243 every 8 connection(s)
I20260619 12:28:30.108201 25161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:30.109824 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 25161
I20260619 12:28:30.109978 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:30.110283 7071 external_mini_cluster.cc:1428] Setting key 0edac45268f2a0ee7cd360dd9a9de18a
I20260619 12:28:30.111446 25229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:30.118351 25229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26: Bootstrap starting.
Jun 19 12:28:30 dist-test-slave-kr9h krb5kdc[25148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872109, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:28:30.120793 25229 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:30.121783 25229 log.cc:824] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:30.124106 25229 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26: No bootstrap required, opened a new log
I20260619 12:28:30.126196 25232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.111827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43066 (local address 127.6.231.254:36243)
0619 12:28:30.112265 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:30.112285 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:30.112329 (+ 44us) server_negotiation.cc:407] Connection header received
0619 12:28:30.113186 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:30.113210 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:30.113671 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:30.114069 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:30.114789 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.116279 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.117168 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.117878 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.118057 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.120606 (+ 2549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:30.120614 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:30.120633 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:30.120669 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:30.122771 (+ 2102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:30.123200 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:30.123205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:30.123211 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:30.123354 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:30.123614 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:30.123617 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:30.123618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:30.124138 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:30.124318 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:30.124636 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 12:28:30.124954 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":120,"threads_started":1}
I20260619 12:28:30.128036 25229 raft_consensus.cc:348] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74358f468694f6c982ddf468a8e5d26" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36243 } }
I20260619 12:28:30.128314 25229 raft_consensus.cc:374] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:30.128413 25229 raft_consensus.cc:729] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c74358f468694f6c982ddf468a8e5d26, State: Initialized, Role: FOLLOWER
I20260619 12:28:30.128926 25229 consensus_queue.cc:260] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74358f468694f6c982ddf468a8e5d26" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36243 } }
I20260619 12:28:30.129057 25229 raft_consensus.cc:388] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:30.129145 25229 raft_consensus.cc:482] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:30.129257 25229 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:30.130262 25229 raft_consensus.cc:504] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74358f468694f6c982ddf468a8e5d26" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36243 } }
I20260619 12:28:30.130630 25229 leader_election.cc:302] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c74358f468694f6c982ddf468a8e5d26; no voters:
I20260619 12:28:30.130952 25229 leader_election.cc:288] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:30.131130 25234 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:30.131372 25234 raft_consensus.cc:686] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [term 1 LEADER]: Becoming Leader. State: Replica: c74358f468694f6c982ddf468a8e5d26, State: Running, Role: LEADER
I20260619 12:28:30.131731 25234 consensus_queue.cc:237] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74358f468694f6c982ddf468a8e5d26" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36243 } }
I20260619 12:28:30.132232 25229 sys_catalog.cc:564] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:30.133471 25235 sys_catalog.cc:455] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c74358f468694f6c982ddf468a8e5d26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74358f468694f6c982ddf468a8e5d26" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36243 } } }
I20260619 12:28:30.133612 25235 sys_catalog.cc:458] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:30.133882 25236 sys_catalog.cc:455] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c74358f468694f6c982ddf468a8e5d26. Latest consensus state: current_term: 1 leader_uuid: "c74358f468694f6c982ddf468a8e5d26" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c74358f468694f6c982ddf468a8e5d26" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 36243 } } }
I20260619 12:28:30.133975 25236 sys_catalog.cc:458] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:30.134776 25243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:30.137584 25243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:30.143985 25243 catalog_manager.cc:1347] Generated new cluster ID: c764d3521e9d44fbbb5409b2c8f808b2
I20260619 12:28:30.144068 25243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:30.157903 25243 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:30.159245 25243 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:30.171942 25243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26: Generated new TSK 0
I20260619 12:28:30.172896 25243 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:28:30.204345 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36243
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:46589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:30.319682 25255 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:30.320030 25255 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:30.320077 25255 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:30.324353 25255 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:30.324440 25255 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:30.324536 25255 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:28:30.330312 25255 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:46589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25255
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:30.331465 25255 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:30.332647 25255 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:30.342958 25260 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:30.342972 25261 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:30.342968 25263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:30.343318 25255 server_base.cc:1034] running on GCE node
I20260619 12:28:30.343811 25255 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:30.344544 25255 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:30.345731 25255 hybrid_clock.cc:648] HybridClock initialized: now 1781872110345671 us; error 73 us; skew 500 ppm
Jun 19 12:28:30 dist-test-slave-kr9h krb5kdc[25148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872110, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:30.348678 25255 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:28:30.349759 25255 webserver.cc:460] Webserver started at http://127.6.231.193:33047/ using document root <none> and password file <none>
I20260619 12:28:30.350394 25255 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:30.350490 25255 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:30.350734 25255 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:30.353262 25255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "116e386b348f45d1a62190a83b9aa160"
format_stamp: "Formatted at 2026-06-19 12:28:30 on dist-test-slave-kr9h"
server_key: "ff1436c8dfee01e7dda1bac50ad928b7"
server_key_iv: "5656de648c675e054ec9bd2866951a72"
server_key_version: "encryptionkey@0"
I20260619 12:28:30.353942 25255 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "116e386b348f45d1a62190a83b9aa160"
format_stamp: "Formatted at 2026-06-19 12:28:30 on dist-test-slave-kr9h"
server_key: "ff1436c8dfee01e7dda1bac50ad928b7"
server_key_iv: "5656de648c675e054ec9bd2866951a72"
server_key_version: "encryptionkey@0"
I20260619 12:28:30.357933 25255 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:28:30.360601 25270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:30.362129 25255 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:30.362282 25255 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "116e386b348f45d1a62190a83b9aa160"
format_stamp: "Formatted at 2026-06-19 12:28:30 on dist-test-slave-kr9h"
server_key: "ff1436c8dfee01e7dda1bac50ad928b7"
server_key_iv: "5656de648c675e054ec9bd2866951a72"
server_key_version: "encryptionkey@0"
I20260619 12:28:30.362407 25255 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:30.381304 25255 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:30.384727 25255 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:30.384958 25255 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:30.385651 25255 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:30.386451 25255 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:30.386498 25255 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:30.386538 25255 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:30.386552 25255 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:30.395718 25255 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:44057
I20260619 12:28:30.395805 25383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:44057 every 8 connection(s)
I20260619 12:28:30.396699 25255 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:28:30.401208 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 25255
I20260619 12:28:30.401360 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:28:30.401680 7071 external_mini_cluster.cc:1428] Setting key d53e1ce2f5c42bcdf78b90ef20f3029d
Jun 19 12:28:30 dist-test-slave-kr9h krb5kdc[25148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872110, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
I20260619 12:28:30.410866 25232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.398692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:38019 (local address 127.6.231.254:36243)
0619 12:28:30.398839 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:30.398843 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:30.399713 (+ 870us) server_negotiation.cc:407] Connection header received
0619 12:28:30.401111 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:30.401114 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:30.401176 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:30.401268 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:30.402605 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.403151 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.404301 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.404538 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.404649 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.407158 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:30.407161 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:30.407163 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:30.407187 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:30.408682 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:30.409212 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:30.409219 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:30.409223 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:30.409307 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:30.409706 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:30.409711 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:30.409712 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:30.409905 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:30.410038 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:30.410559 (+ 521us) server_negotiation.cc:299] Negotiation successful
0619 12:28:30.410700 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 12:28:30.411720 25386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.399106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36243 (local address 127.6.231.193:38019)
0619 12:28:30.399561 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:30.399604 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:30.400828 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:30.401511 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:30.401521 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:30.401942 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:30.402446 (+ 504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:30.402458 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.403288 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.403292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:30.404175 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:30.404184 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.404691 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.404695 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:30.404875 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.405540 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:28:30.405558 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:30.406978 (+ 1420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:30.408807 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:30.408813 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:30.408831 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:30.409106 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:30.409401 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:30.409405 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:30.409408 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:30.409607 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:30.410023 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:30.410031 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:30.410308 (+ 277us) client_negotiation.cc:769] Sending connection context
0619 12:28:30.410514 (+ 206us) client_negotiation.cc:240] Negotiation successful
0619 12:28:30.410872 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":119,"threads_started":1}
I20260619 12:28:30.412968 25384 heartbeater.cc:347] Connected to a master server at 127.6.231.254:36243
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:28:30.413300 25384 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:30.413988 25384 heartbeater.cc:511] Master 127.6.231.254:36243 requested a full tablet report, sending...
I20260619 12:28:30.415886 25193 ts_manager.cc:194] Registered new tserver with Master: 116e386b348f45d1a62190a83b9aa160 (127.6.231.193:44057)
I20260619 12:28:30.417160 25193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:38019
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:28:30.430541 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36243
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:46589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:30.545408 25389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:30.545749 25389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:30.545795 25389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:30.550052 25389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:30.550127 25389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:30.550189 25389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:28:30.555656 25389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:46589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25389
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:30.556862 25389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:30.557924 25389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:30.567808 25397 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:30.567833 25395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:30.567817 25394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:30.568398 25389 server_base.cc:1034] running on GCE node
I20260619 12:28:30.568815 25389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:30.569505 25389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:30.570698 25389 hybrid_clock.cc:648] HybridClock initialized: now 1781872110570640 us; error 95 us; skew 500 ppm
Jun 19 12:28:30 dist-test-slave-kr9h krb5kdc[25148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872110, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:30.574028 25389 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:28:30.575052 25389 webserver.cc:460] Webserver started at http://127.6.231.194:41977/ using document root <none> and password file <none>
I20260619 12:28:30.575724 25389 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:30.575798 25389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:30.575984 25389 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:30.578264 25389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "17e4e716aed6440d918cdcb8184a8bf3"
format_stamp: "Formatted at 2026-06-19 12:28:30 on dist-test-slave-kr9h"
server_key: "79d8f639bc85c6371d065d39015619f5"
server_key_iv: "e4e6e133e40688f88ef19649cf4bd22b"
server_key_version: "encryptionkey@0"
I20260619 12:28:30.578835 25389 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "17e4e716aed6440d918cdcb8184a8bf3"
format_stamp: "Formatted at 2026-06-19 12:28:30 on dist-test-slave-kr9h"
server_key: "79d8f639bc85c6371d065d39015619f5"
server_key_iv: "e4e6e133e40688f88ef19649cf4bd22b"
server_key_version: "encryptionkey@0"
I20260619 12:28:30.583003 25389 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:28:30.585706 25404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:30.587146 25389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:30.587293 25389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "17e4e716aed6440d918cdcb8184a8bf3"
format_stamp: "Formatted at 2026-06-19 12:28:30 on dist-test-slave-kr9h"
server_key: "79d8f639bc85c6371d065d39015619f5"
server_key_iv: "e4e6e133e40688f88ef19649cf4bd22b"
server_key_version: "encryptionkey@0"
I20260619 12:28:30.587423 25389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:30.600001 25389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:30.603444 25389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:30.603603 25389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:30.604151 25389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:30.604954 25389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:30.605003 25389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:30.605042 25389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:30.605067 25389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:30.614851 25389 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:34027
I20260619 12:28:30.614867 25517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:34027 every 8 connection(s)
I20260619 12:28:30.615837 25389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:28:30.617733 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 25389
I20260619 12:28:30.617892 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:28:30.618212 7071 external_mini_cluster.cc:1428] Setting key 53f2dc1396afec1d372c77132b7c33df
Jun 19 12:28:30 dist-test-slave-kr9h krb5kdc[25148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872110, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
I20260619 12:28:30.634737 25232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.618493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:35431 (local address 127.6.231.254:36243)
0619 12:28:30.618656 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:30.618661 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:30.619821 (+ 1160us) server_negotiation.cc:407] Connection header received
0619 12:28:30.622041 (+ 2220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:30.622044 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:30.622156 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:30.622277 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:30.623763 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.624669 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.626098 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.626537 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.626671 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.629723 (+ 3052us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:30.629727 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:30.629729 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:30.629785 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:30.631441 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:30.632148 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:30.632151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:30.632152 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:30.632416 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:30.632897 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:30.632900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:30.632901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:30.633123 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:30.633272 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:30.634339 (+ 1067us) server_negotiation.cc:299] Negotiation successful
0619 12:28:30.634522 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260619 12:28:30.636153 25520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.618952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36243 (local address 127.6.231.194:35431)
0619 12:28:30.619530 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:30.619578 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:30.621755 (+ 2177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:30.622498 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:30.622507 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:30.623033 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:30.623598 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:30.623613 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.624857 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.624861 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:30.625955 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:30.625965 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.626714 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.626716 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:30.626865 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.627514 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:28:30.627531 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:30.629355 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:30.631582 (+ 2227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:30.631588 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:30.631636 (+ 48us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:30.632002 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:30.632541 (+ 539us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:30.632545 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:30.632546 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:30.632721 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:30.633290 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:30.633300 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:30.634000 (+ 700us) client_negotiation.cc:769] Sending connection context
0619 12:28:30.634236 (+ 236us) client_negotiation.cc:240] Negotiation successful
0619 12:28:30.634740 (+ 504us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":179,"threads_started":1}
I20260619 12:28:30.637612 25518 heartbeater.cc:347] Connected to a master server at 127.6.231.254:36243
I20260619 12:28:30.637899 25518 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:30.638443 25518 heartbeater.cc:511] Master 127.6.231.254:36243 requested a full tablet report, sending...
I20260619 12:28:30.639624 25193 ts_manager.cc:194] Registered new tserver with Master: 17e4e716aed6440d918cdcb8184a8bf3 (127.6.231.194:34027)
I20260619 12:28:30.640270 25193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:35431
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:28:30.650367 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36243
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:46589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:30.765170 25523 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:30.765514 25523 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:30.765616 25523 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:30.769924 25523 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:30.770036 25523 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:30.770133 25523 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:28:30.775633 25523 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:46589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:36243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25523
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:30.776765 25523 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:30.777875 25523 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:30.787810 25531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:30.787950 25523 server_base.cc:1034] running on GCE node
W20260619 12:28:30.788080 25529 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:30.787814 25528 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:30.788865 25523 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:30.789691 25523 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:30.790853 25523 hybrid_clock.cc:648] HybridClock initialized: now 1781872110790827 us; error 41 us; skew 500 ppm
Jun 19 12:28:30 dist-test-slave-kr9h krb5kdc[25148](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872110, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:30.793891 25523 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:28:30.794809 25523 webserver.cc:460] Webserver started at http://127.6.231.195:45353/ using document root <none> and password file <none>
I20260619 12:28:30.795392 25523 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:30.795452 25523 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:30.795691 25523 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:30.797925 25523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "0950bdf8073f400c8466a20105a40ac2"
format_stamp: "Formatted at 2026-06-19 12:28:30 on dist-test-slave-kr9h"
server_key: "ee47f763dcd51adefb3c56be6890f6a4"
server_key_iv: "1ee66aef7c59a43ab5432d3421948bce"
server_key_version: "encryptionkey@0"
I20260619 12:28:30.798511 25523 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "0950bdf8073f400c8466a20105a40ac2"
format_stamp: "Formatted at 2026-06-19 12:28:30 on dist-test-slave-kr9h"
server_key: "ee47f763dcd51adefb3c56be6890f6a4"
server_key_iv: "1ee66aef7c59a43ab5432d3421948bce"
server_key_version: "encryptionkey@0"
I20260619 12:28:30.802474 25523 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 12:28:30.805154 25538 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:30.806417 25523 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 12:28:30.806538 25523 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "0950bdf8073f400c8466a20105a40ac2"
format_stamp: "Formatted at 2026-06-19 12:28:30 on dist-test-slave-kr9h"
server_key: "ee47f763dcd51adefb3c56be6890f6a4"
server_key_iv: "1ee66aef7c59a43ab5432d3421948bce"
server_key_version: "encryptionkey@0"
I20260619 12:28:30.806651 25523 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:30.821554 25523 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:30.824641 25523 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:30.824815 25523 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:30.825387 25523 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:30.826258 25523 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:30.826303 25523 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:30.826350 25523 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:30.826366 25523 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:30.836370 25523 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:40271
I20260619 12:28:30.836388 25651 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:40271 every 8 connection(s)
I20260619 12:28:30.837355 25523 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:28:30.837621 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 25523
I20260619 12:28:30.837764 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:28:30.838120 7071 external_mini_cluster.cc:1428] Setting key c46ddd49f6ff30f4d1167c9442badc8e
Jun 19 12:28:30 dist-test-slave-kr9h krb5kdc[25148](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872110, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:28:30.852356 25232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.839451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:51293 (local address 127.6.231.254:36243)
0619 12:28:30.839614 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:30.839619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:30.840782 (+ 1163us) server_negotiation.cc:407] Connection header received
0619 12:28:30.842164 (+ 1382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:30.842166 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:30.842220 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:30.842303 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:30.843550 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.844095 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.845298 (+ 1203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.845595 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.845717 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.848277 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:30.848281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:30.848284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:30.848320 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:30.849897 (+ 1577us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:30.850450 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:30.850454 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:30.850456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:30.850542 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:30.850972 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:30.850976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:30.850977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:30.851245 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:30.851360 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:30.852055 (+ 695us) server_negotiation.cc:299] Negotiation successful
0619 12:28:30.852193 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260619 12:28:30.853308 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.839999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:36243 (local address 127.6.231.195:51293)
0619 12:28:30.840611 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:30.840674 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:30.841947 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:30.842503 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:30.842510 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:30.842889 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:30.843395 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:30.843408 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.844225 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.844229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:30.845169 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:30.845179 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.845760 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.845764 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:30.845946 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.846603 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:28:30.846619 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:30.848099 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:30.850044 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:30.850050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:30.850068 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:30.850341 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:30.850643 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:30.850646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:30.850648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:30.850830 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:30.851359 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:30.851365 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:30.851730 (+ 365us) client_negotiation.cc:769] Sending connection context
0619 12:28:30.851985 (+ 255us) client_negotiation.cc:240] Negotiation successful
0619 12:28:30.852333 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":162,"threads_started":1}
I20260619 12:28:30.854497 25652 heartbeater.cc:347] Connected to a master server at 127.6.231.254:36243
I20260619 12:28:30.854738 25652 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:30.855264 25652 heartbeater.cc:511] Master 127.6.231.254:36243 requested a full tablet report, sending...
I20260619 12:28:30.856313 25193 ts_manager.cc:194] Registered new tserver with Master: 0950bdf8073f400c8466a20105a40ac2 (127.6.231.195:40271)
I20260619 12:28:30.856895 25193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:51293
I20260619 12:28:30.864796 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:28:30.875315 25232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.866904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43078 (local address 127.6.231.254:36243)
0619 12:28:30.867063 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:30.867068 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:30.867243 (+ 175us) server_negotiation.cc:407] Connection header received
0619 12:28:30.867366 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:30.867368 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:30.867413 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:30.867486 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:30.868198 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.868713 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.869638 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.869849 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.869930 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.871591 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:30.871594 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:30.871596 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:30.871621 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:30.873211 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:30.873868 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:30.873870 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:30.873872 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:30.873949 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:30.874276 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:30.874279 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:30.874280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:30.874507 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:30.874629 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:30.874989 (+ 360us) server_negotiation.cc:299] Negotiation successful
0619 12:28:30.875122 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 12:28:30.878638 25193 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43078:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:28:30.881714 25193 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:28:30.897356 25663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.890678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:34027 (local address 127.0.0.1:60826)
0619 12:28:30.891124 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:30.891184 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:30.891330 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:30.892577 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:30.892582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:30.892604 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:30.892774 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:30.892781 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.894391 (+ 1610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.894395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:30.895901 (+ 1506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:30.895910 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.896791 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.896794 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:30.896897 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.896931 (+ 34us) client_negotiation.cc:769] Sending connection context
0619 12:28:30.897054 (+ 123us) client_negotiation.cc:240] Negotiation successful
0619 12:28:30.897180 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"mutex_wait_us":101,"spinlock_wait_cycles":5376,"thread_start_us":80,"threads_started":1}
I20260619 12:28:30.897884 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.891590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60826 (local address 127.6.231.194:34027)
0619 12:28:30.892055 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:30.892069 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:30.892093 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:28:30.892171 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:30.892176 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:30.892369 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:30.892584 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:30.892896 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.894156 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.896072 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.896660 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.896781 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.896939 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:30.897290 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 12:28:30.897405 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":142,"threads_started":1}
I20260619 12:28:30.897775 25668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.892163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34764 (local address 127.6.231.193:44057)
0619 12:28:30.892457 (+ 294us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:30.892473 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:30.892761 (+ 288us) server_negotiation.cc:407] Connection header received
0619 12:28:30.892848 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:30.892856 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:30.893062 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:30.893224 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:30.893488 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.894501 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.896087 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.896940 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.897040 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.897177 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:30.897517 (+ 340us) server_negotiation.cc:299] Negotiation successful
0619 12:28:30.897629 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":109,"threads_started":1}
I20260619 12:28:30.897408 25667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.891662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:44057 (local address 127.0.0.1:34764)
0619 12:28:30.892547 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:30.892562 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:30.892664 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:30.893200 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:30.893203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:30.893221 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:30.893323 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:30.893330 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.894670 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.894674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:30.895946 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:30.895955 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.897041 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.897048 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:30.897135 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.897154 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 12:28:30.897241 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 12:28:30.897294 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":814,"mutex_wait_us":11,"spinlock_wait_cycles":1664,"thread_start_us":112,"threads_started":1}
I20260619 12:28:30.898757 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.890828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41540 (local address 127.6.231.195:40271)
0619 12:28:30.891231 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:30.891245 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:30.891715 (+ 470us) server_negotiation.cc:407] Connection header received
0619 12:28:30.891967 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:30.891974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:30.892234 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:30.892403 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:30.892952 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.894158 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.896886 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.897724 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.897836 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.897986 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:30.898510 (+ 524us) server_negotiation.cc:299] Negotiation successful
0619 12:28:30.898621 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":171,"threads_started":1}
I20260619 12:28:30.899188 25665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:30.890677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:40271 (local address 127.0.0.1:41540)
0619 12:28:30.891615 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:30.891629 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:30.891796 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:30.892651 (+ 855us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:30.892654 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:30.892669 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:30.892810 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:30.892816 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.895593 (+ 2777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.895596 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:30.896716 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:30.896727 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:30.898202 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:30.898207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:30.898344 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:30.898371 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 12:28:30.898959 (+ 588us) client_negotiation.cc:240] Negotiation successful
0619 12:28:30.899037 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":872,"mutex_wait_us":469,"thread_start_us":91,"threads_started":1}
I20260619 12:28:30.901218 25452 tablet_service.cc:1467] Processing CreateTablet for tablet 3e370d9df2734c6b87a0b20da719eace (DEFAULT_TABLE table=test-table [id=117c53bfc92e4986835dc6fb3690b2df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:30.901335 25318 tablet_service.cc:1467] Processing CreateTablet for tablet 3e370d9df2734c6b87a0b20da719eace (DEFAULT_TABLE table=test-table [id=117c53bfc92e4986835dc6fb3690b2df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:30.901841 25586 tablet_service.cc:1467] Processing CreateTablet for tablet 3e370d9df2734c6b87a0b20da719eace (DEFAULT_TABLE table=test-table [id=117c53bfc92e4986835dc6fb3690b2df]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:30.902465 25452 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e370d9df2734c6b87a0b20da719eace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:30.902465 25318 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e370d9df2734c6b87a0b20da719eace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:30.902768 25586 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3e370d9df2734c6b87a0b20da719eace. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:30.909420 25669 tablet_bootstrap.cc:492] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3: Bootstrap starting.
I20260619 12:28:30.910552 25670 tablet_bootstrap.cc:492] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160: Bootstrap starting.
I20260619 12:28:30.910899 25671 tablet_bootstrap.cc:492] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2: Bootstrap starting.
I20260619 12:28:30.912621 25670 tablet_bootstrap.cc:654] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:30.913030 25671 tablet_bootstrap.cc:654] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:30.913496 25669 tablet_bootstrap.cc:654] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:30.913511 25670 log.cc:824] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:30.914336 25671 log.cc:824] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:30.914337 25669 log.cc:824] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:30.915607 25670 tablet_bootstrap.cc:492] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160: No bootstrap required, opened a new log
I20260619 12:28:30.915834 25670 ts_tablet_manager.cc:1397] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 12:28:30.916072 25669 tablet_bootstrap.cc:492] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3: No bootstrap required, opened a new log
I20260619 12:28:30.916214 25669 ts_tablet_manager.cc:1397] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260619 12:28:30.918865 25671 tablet_bootstrap.cc:492] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2: No bootstrap required, opened a new log
I20260619 12:28:30.919056 25671 ts_tablet_manager.cc:1397] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260619 12:28:30.919314 25670 raft_consensus.cc:348] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0950bdf8073f400c8466a20105a40ac2" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40271 } } peers { permanent_uuid: "17e4e716aed6440d918cdcb8184a8bf3" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34027 } } peers { permanent_uuid: "116e386b348f45d1a62190a83b9aa160" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 44057 } }
I20260619 12:28:30.919589 25670 raft_consensus.cc:374] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:30.919642 25670 raft_consensus.cc:729] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 116e386b348f45d1a62190a83b9aa160, State: Initialized, Role: FOLLOWER
I20260619 12:28:30.919628 25669 raft_consensus.cc:348] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0950bdf8073f400c8466a20105a40ac2" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40271 } } peers { permanent_uuid: "17e4e716aed6440d918cdcb8184a8bf3" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34027 } } peers { permanent_uuid: "116e386b348f45d1a62190a83b9aa160" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 44057 } }
I20260619 12:28:30.919888 25669 raft_consensus.cc:374] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:30.919941 25669 raft_consensus.cc:729] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17e4e716aed6440d918cdcb8184a8bf3, State: Initialized, Role: FOLLOWER
I20260619 12:28:30.920377 25669 consensus_queue.cc:260] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0950bdf8073f400c8466a20105a40ac2" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40271 } } peers { permanent_uuid: "17e4e716aed6440d918cdcb8184a8bf3" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34027 } } peers { permanent_uuid: "116e386b348f45d1a62190a83b9aa160" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 44057 } }
I20260619 12:28:30.920375 25670 consensus_queue.cc:260] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0950bdf8073f400c8466a20105a40ac2" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40271 } } peers { permanent_uuid: "17e4e716aed6440d918cdcb8184a8bf3" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34027 } } peers { permanent_uuid: "116e386b348f45d1a62190a83b9aa160" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 44057 } }
I20260619 12:28:30.921382 25670 ts_tablet_manager.cc:1428] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260619 12:28:30.921389 25669 ts_tablet_manager.cc:1428] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3: Time spent starting tablet: real 0.005s user 0.005s sys 0.001s
I20260619 12:28:30.921770 25384 heartbeater.cc:503] Master 127.6.231.254:36243 was elected leader, sending a full tablet report...
I20260619 12:28:30.922132 25518 heartbeater.cc:503] Master 127.6.231.254:36243 was elected leader, sending a full tablet report...
I20260619 12:28:30.922960 25671 raft_consensus.cc:348] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0950bdf8073f400c8466a20105a40ac2" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40271 } } peers { permanent_uuid: "17e4e716aed6440d918cdcb8184a8bf3" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34027 } } peers { permanent_uuid: "116e386b348f45d1a62190a83b9aa160" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 44057 } }
I20260619 12:28:30.923254 25671 raft_consensus.cc:374] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:30.923309 25671 raft_consensus.cc:729] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0950bdf8073f400c8466a20105a40ac2, State: Initialized, Role: FOLLOWER
I20260619 12:28:30.923743 25671 consensus_queue.cc:260] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0950bdf8073f400c8466a20105a40ac2" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40271 } } peers { permanent_uuid: "17e4e716aed6440d918cdcb8184a8bf3" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34027 } } peers { permanent_uuid: "116e386b348f45d1a62190a83b9aa160" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 44057 } }
I20260619 12:28:30.924561 25652 heartbeater.cc:503] Master 127.6.231.254:36243 was elected leader, sending a full tablet report...
I20260619 12:28:30.924803 25671 ts_tablet_manager.cc:1428] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
W20260619 12:28:31.088526 25653 tablet.cc:2369] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:28:31.117385 25519 tablet.cc:2369] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:28:31.119660 25677 raft_consensus.cc:482] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:28:31.119828 25677 raft_consensus.cc:504] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0950bdf8073f400c8466a20105a40ac2" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40271 } } peers { permanent_uuid: "17e4e716aed6440d918cdcb8184a8bf3" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34027 } } peers { permanent_uuid: "116e386b348f45d1a62190a83b9aa160" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 44057 } }
I20260619 12:28:31.120915 25677 leader_election.cc:288] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17e4e716aed6440d918cdcb8184a8bf3 (127.6.231.194:34027), 116e386b348f45d1a62190a83b9aa160 (127.6.231.193:44057)
I20260619 12:28:31.125535 25654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:31.121263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:44057 (local address 127.6.231.195:55009)
0619 12:28:31.121394 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:31.121414 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:31.121583 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:31.121914 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:31.121917 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:31.121951 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:31.122079 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:31.122084 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.123141 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.123145 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:31.124098 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:31.124107 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.125026 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.125030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:31.125147 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:31.125174 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 12:28:31.125263 (+ 89us) client_negotiation.cc:240] Negotiation successful
0619 12:28:31.125332 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260619 12:28:31.125573 25668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:31.121470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:55009 (local address 127.6.231.193:44057)
0619 12:28:31.121623 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:31.121627 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:31.121641 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:28:31.121693 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:31.121695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:31.121748 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:31.121907 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:31.122254 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.123012 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.124255 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.124892 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.124997 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:31.125050 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:31.125314 (+ 264us) server_negotiation.cc:299] Negotiation successful
0619 12:28:31.125419 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 12:28:31.125972 25666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:31.121475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:53631 (local address 127.6.231.194:34027)
0619 12:28:31.121644 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:31.121650 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:31.122153 (+ 503us) server_negotiation.cc:407] Connection header received
0619 12:28:31.122275 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:31.122277 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:31.122328 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:31.122387 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:31.122733 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.123315 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.124694 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.125338 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.125486 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:31.125532 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:31.125774 (+ 242us) server_negotiation.cc:299] Negotiation successful
0619 12:28:31.125860 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260619 12:28:31.126339 25678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:31.121263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:34027 (local address 127.6.231.195:53631)
0619 12:28:31.122048 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:31.122060 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:31.122149 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:31.122431 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:31.122435 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:31.122453 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:31.122572 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:31.122579 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.123457 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.123459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:31.124530 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:31.124540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.125498 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.125502 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:31.125610 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:31.125638 (+ 28us) client_negotiation.cc:769] Sending connection context
0619 12:28:31.126124 (+ 486us) client_negotiation.cc:240] Negotiation successful
0619 12:28:31.126198 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":715,"mutex_wait_us":401,"thread_start_us":149,"threads_started":1}
I20260619 12:28:31.126368 25338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e370d9df2734c6b87a0b20da719eace" candidate_uuid: "0950bdf8073f400c8466a20105a40ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116e386b348f45d1a62190a83b9aa160" is_pre_election: true
I20260619 12:28:31.126762 25338 raft_consensus.cc:2454] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0950bdf8073f400c8466a20105a40ac2 in term 0.
I20260619 12:28:31.127035 25472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e370d9df2734c6b87a0b20da719eace" candidate_uuid: "0950bdf8073f400c8466a20105a40ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e4e716aed6440d918cdcb8184a8bf3" is_pre_election: true
I20260619 12:28:31.127364 25472 raft_consensus.cc:2454] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0950bdf8073f400c8466a20105a40ac2 in term 0.
I20260619 12:28:31.127379 25542 leader_election.cc:302] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0950bdf8073f400c8466a20105a40ac2, 116e386b348f45d1a62190a83b9aa160; no voters:
I20260619 12:28:31.127709 25677 raft_consensus.cc:2781] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:28:31.127812 25677 raft_consensus.cc:482] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:28:31.127957 25677 raft_consensus.cc:3037] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:31.129089 25677 raft_consensus.cc:504] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0950bdf8073f400c8466a20105a40ac2" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40271 } } peers { permanent_uuid: "17e4e716aed6440d918cdcb8184a8bf3" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34027 } } peers { permanent_uuid: "116e386b348f45d1a62190a83b9aa160" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 44057 } }
I20260619 12:28:31.129562 25677 leader_election.cc:288] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [CANDIDATE]: Term 1 election: Requested vote from peers 17e4e716aed6440d918cdcb8184a8bf3 (127.6.231.194:34027), 116e386b348f45d1a62190a83b9aa160 (127.6.231.193:44057)
I20260619 12:28:31.129905 25472 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e370d9df2734c6b87a0b20da719eace" candidate_uuid: "0950bdf8073f400c8466a20105a40ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17e4e716aed6440d918cdcb8184a8bf3"
I20260619 12:28:31.130007 25338 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3e370d9df2734c6b87a0b20da719eace" candidate_uuid: "0950bdf8073f400c8466a20105a40ac2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "116e386b348f45d1a62190a83b9aa160"
I20260619 12:28:31.130065 25472 raft_consensus.cc:3037] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:31.130154 25338 raft_consensus.cc:3037] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:31.131359 25472 raft_consensus.cc:2454] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0950bdf8073f400c8466a20105a40ac2 in term 1.
I20260619 12:28:31.131414 25338 raft_consensus.cc:2454] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0950bdf8073f400c8466a20105a40ac2 in term 1.
I20260619 12:28:31.131728 25542 leader_election.cc:302] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0950bdf8073f400c8466a20105a40ac2, 116e386b348f45d1a62190a83b9aa160; no voters:
I20260619 12:28:31.131929 25677 raft_consensus.cc:2781] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:31.132177 25677 raft_consensus.cc:686] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [term 1 LEADER]: Becoming Leader. State: Replica: 0950bdf8073f400c8466a20105a40ac2, State: Running, Role: LEADER
I20260619 12:28:31.132576 25677 consensus_queue.cc:237] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0950bdf8073f400c8466a20105a40ac2" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40271 } } peers { permanent_uuid: "17e4e716aed6440d918cdcb8184a8bf3" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34027 } } peers { permanent_uuid: "116e386b348f45d1a62190a83b9aa160" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 44057 } }
I20260619 12:28:31.136191 25193 catalog_manager.cc:5495] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0950bdf8073f400c8466a20105a40ac2 (127.6.231.195). New cstate: current_term: 1 leader_uuid: "0950bdf8073f400c8466a20105a40ac2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0950bdf8073f400c8466a20105a40ac2" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 40271 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "17e4e716aed6440d918cdcb8184a8bf3" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "116e386b348f45d1a62190a83b9aa160" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 44057 } health_report { overall_health: UNKNOWN } } }
W20260619 12:28:31.148432 25385 tablet.cc:2369] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:28:31.194105 25664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:31.190334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41552 (local address 127.6.231.195:40271)
0619 12:28:31.190482 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:31.190485 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:31.190520 (+ 35us) server_negotiation.cc:407] Connection header received
0619 12:28:31.190590 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:31.190592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:31.190656 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:31.190744 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:28:31.191138 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.191654 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.192594 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.192812 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.192904 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:31.193189 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:28:31.193633 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:28:31.193746 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:31.193952 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 12:28:31.193990 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 12:28:31.200260 25472 raft_consensus.cc:1261] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3 [term 1 FOLLOWER]: Refusing update from remote peer 0950bdf8073f400c8466a20105a40ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:28:31.200304 25338 raft_consensus.cc:1261] T 3e370d9df2734c6b87a0b20da719eace P 116e386b348f45d1a62190a83b9aa160 [term 1 FOLLOWER]: Refusing update from remote peer 0950bdf8073f400c8466a20105a40ac2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:28:31.201073 25679 consensus_queue.cc:1035] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "17e4e716aed6440d918cdcb8184a8bf3" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:28:31.201304 25677 consensus_queue.cc:1035] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "116e386b348f45d1a62190a83b9aa160" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 44057 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:28:31.209059 25684 mvcc.cc:205] Tried to move back new op lower bound from 7298548167468609536 to 7298548167201968128. Current Snapshot: MvccSnapshot[applied={T|T < 7298548167468609536}]
I20260619 12:28:31.210150 25683 mvcc.cc:205] Tried to move back new op lower bound from 7298548167468609536 to 7298548167201968128. Current Snapshot: MvccSnapshot[applied={T|T < 7298548167468609536}]
I20260619 12:28:31.218487 25193 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:28:31.218700 25193 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43078:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:28:31.221572 25193 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c74358f468694f6c982ddf468a8e5d26: Sending DeleteTablet for 3 replicas of tablet 3e370d9df2734c6b87a0b20da719eace
I20260619 12:28:31.222422 25452 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e370d9df2734c6b87a0b20da719eace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:31 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:60826
I20260619 12:28:31.222433 25586 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e370d9df2734c6b87a0b20da719eace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:31 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:41540
I20260619 12:28:31.222609 25318 tablet_service.cc:1514] Processing DeleteTablet for tablet 3e370d9df2734c6b87a0b20da719eace with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:31 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:34764
I20260619 12:28:31.223057 25692 tablet_replica.cc:330] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2: stopping tablet replica
I20260619 12:28:31.223114 25691 tablet_replica.cc:330] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3: stopping tablet replica
I20260619 12:28:31.223403 25692 raft_consensus.cc:2229] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:28:31.223430 25691 raft_consensus.cc:2229] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:28:31.223774 25691 raft_consensus.cc:2258] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:28:31.223918 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 25255
I20260619 12:28:31.223805 25692 raft_consensus.cc:2258] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:28:31.226356 25692 ts_tablet_manager.cc:1905] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:28:31.226956 25691 ts_tablet_manager.cc:1905] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:28:31.229807 25692 ts_tablet_manager.cc:1918] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:28:31.230051 25692 log.cc:1196] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/3e370d9df2734c6b87a0b20da719eace
I20260619 12:28:31.230098 25691 ts_tablet_manager.cc:1918] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:28:31.230193 25691 log.cc:1196] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/3e370d9df2734c6b87a0b20da719eace
I20260619 12:28:31.230561 25692 ts_tablet_manager.cc:1939] T 3e370d9df2734c6b87a0b20da719eace P 0950bdf8073f400c8466a20105a40ac2: Deleting consensus metadata
I20260619 12:28:31.230623 25691 ts_tablet_manager.cc:1939] T 3e370d9df2734c6b87a0b20da719eace P 17e4e716aed6440d918cdcb8184a8bf3: Deleting consensus metadata
I20260619 12:28:31.233945 25178 catalog_manager.cc:4841] TS 17e4e716aed6440d918cdcb8184a8bf3 (127.6.231.194:34027): tablet 3e370d9df2734c6b87a0b20da719eace (table test-table [id=117c53bfc92e4986835dc6fb3690b2df]) successfully deleted
W20260619 12:28:31.235260 25180 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:44057 (error 108)
I20260619 12:28:31.235458 25177 catalog_manager.cc:4841] TS 0950bdf8073f400c8466a20105a40ac2 (127.6.231.195:40271): tablet 3e370d9df2734c6b87a0b20da719eace (table test-table [id=117c53bfc92e4986835dc6fb3690b2df]) successfully deleted
I20260619 12:28:31.236203 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 25389
I20260619 12:28:31.238291 25665 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:31.237927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:44057 (local address 127.0.0.1:34766)
0619 12:28:31.238084 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:31.238175 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:44057: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 12:28:31.238653 25180 catalog_manager.cc:4584] TS 116e386b348f45d1a62190a83b9aa160 (127.6.231.193:44057): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3e370d9df2734c6b87a0b20da719eace: Network error: Client connection negotiation failed: client connection to 127.6.231.193:44057: connect: Connection refused (error 111)
I20260619 12:28:31.243186 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 25523
I20260619 12:28:31.249123 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 25161
2026-06-19T12:28:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2496 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-06-19T12:28:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:31Z Disabled control of system clock
I20260619 12:28:31.282100 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:40835
--webserver_interface=127.6.231.254
--webserver_port=0
--builtin_ntp_servers=127.6.231.212:33915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:40835
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:28:31.394420 25698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:31.394742 25698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:31.394819 25698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:31.399034 25698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:31.399155 25698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:31.399185 25698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:31.399227 25698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:31.399267 25698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:31.404336 25698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:33915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:40835
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:40835
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:31.405552 25698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:31.407019 25698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:31.415715 25703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:31.415706 25706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:31.415706 25704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:31.415832 25698 server_base.cc:1034] running on GCE node
I20260619 12:28:31.416636 25698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:31.418071 25698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:31.419245 25698 hybrid_clock.cc:648] HybridClock initialized: now 1781872111419227 us; error 32 us; skew 500 ppm
I20260619 12:28:31.421571 25698 webserver.cc:460] Webserver started at http://127.6.231.254:34905/ using document root <none> and password file <none>
I20260619 12:28:31.422247 25698 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:31.422329 25698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:31.422585 25698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:31.424944 25698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "c46f73338e524f3b9913084278c26d16"
format_stamp: "Formatted at 2026-06-19 12:28:31 on dist-test-slave-kr9h"
server_key: "33a6d2e2c4f877ab1349d8afce3e64ab"
server_key_iv: "26adbda2d32362085c030ccc731e16e3"
server_key_version: "encryptionkey@0"
I20260619 12:28:31.425621 25698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "c46f73338e524f3b9913084278c26d16"
format_stamp: "Formatted at 2026-06-19 12:28:31 on dist-test-slave-kr9h"
server_key: "33a6d2e2c4f877ab1349d8afce3e64ab"
server_key_iv: "26adbda2d32362085c030ccc731e16e3"
server_key_version: "encryptionkey@0"
I20260619 12:28:31.429953 25698 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:28:31.432672 25712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:31.434177 25698 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:31.434324 25698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "c46f73338e524f3b9913084278c26d16"
format_stamp: "Formatted at 2026-06-19 12:28:31 on dist-test-slave-kr9h"
server_key: "33a6d2e2c4f877ab1349d8afce3e64ab"
server_key_iv: "26adbda2d32362085c030ccc731e16e3"
server_key_version: "encryptionkey@0"
I20260619 12:28:31.434444 25698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:31.461647 25698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:31.462414 25698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:31.462628 25698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:31.470085 25698 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:40835
I20260619 12:28:31.470165 25764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:40835 every 8 connection(s)
I20260619 12:28:31.471151 25698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:31.474659 25765 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:31.478466 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 25698
I20260619 12:28:31.478618 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:31.478907 7071 external_mini_cluster.cc:1428] Setting key 198cf8c8eed25d813963f285e4144e81
I20260619 12:28:31.481220 25765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16: Bootstrap starting.
I20260619 12:28:31.483755 25765 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:31.484609 25765 log.cc:824] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:31.486964 25765 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16: No bootstrap required, opened a new log
I20260619 12:28:31.488862 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:31.480443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32896 (local address 127.6.231.254:40835)
0619 12:28:31.481133 (+ 690us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:31.481149 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:31.481186 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:28:31.481931 (+ 745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:31.481952 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:31.482381 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:31.482732 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:31.483190 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.484230 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.485214 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.485940 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.486116 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:31.487025 (+ 909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:31.487033 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:31.487054 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:31.487510 (+ 456us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:31.487570 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:31.487591 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:31.487700 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:31.487828 (+ 128us) server_negotiation.cc:299] Negotiation successful
0619 12:28:31.488117 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":152,"threads_started":1}
I20260619 12:28:31.491570 25765 raft_consensus.cc:348] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46f73338e524f3b9913084278c26d16" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 40835 } }
I20260619 12:28:31.491865 25765 raft_consensus.cc:374] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:31.491957 25765 raft_consensus.cc:729] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c46f73338e524f3b9913084278c26d16, State: Initialized, Role: FOLLOWER
I20260619 12:28:31.492419 25765 consensus_queue.cc:260] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46f73338e524f3b9913084278c26d16" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 40835 } }
I20260619 12:28:31.492584 25765 raft_consensus.cc:388] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:31.492678 25765 raft_consensus.cc:482] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:31.492794 25765 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:31.494076 25765 raft_consensus.cc:504] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46f73338e524f3b9913084278c26d16" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 40835 } }
I20260619 12:28:31.494458 25765 leader_election.cc:302] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c46f73338e524f3b9913084278c26d16; no voters:
I20260619 12:28:31.494812 25765 leader_election.cc:288] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:31.494961 25770 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:31.495229 25770 raft_consensus.cc:686] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [term 1 LEADER]: Becoming Leader. State: Replica: c46f73338e524f3b9913084278c26d16, State: Running, Role: LEADER
I20260619 12:28:31.495548 25770 consensus_queue.cc:237] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46f73338e524f3b9913084278c26d16" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 40835 } }
I20260619 12:28:31.496390 25765 sys_catalog.cc:564] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:31.497349 25771 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c46f73338e524f3b9913084278c26d16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46f73338e524f3b9913084278c26d16" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 40835 } } }
I20260619 12:28:31.497527 25771 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:31.497887 25779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:31.497872 25772 sys_catalog.cc:455] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c46f73338e524f3b9913084278c26d16. Latest consensus state: current_term: 1 leader_uuid: "c46f73338e524f3b9913084278c26d16" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c46f73338e524f3b9913084278c26d16" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 40835 } } }
I20260619 12:28:31.498073 25772 sys_catalog.cc:458] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:31.501189 25779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:31.507637 25779 catalog_manager.cc:1347] Generated new cluster ID: c60ee3aceacf47a69e62827fc528422d
I20260619 12:28:31.507730 25779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:31.520201 25779 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:31.521193 25779 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:31.527729 25779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16: Generated new TSK 0
I20260619 12:28:31.528723 25779 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:31.535001 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:40835
--builtin_ntp_servers=127.6.231.212:33915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:28:31.649102 25789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:31.649506 25789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:31.649588 25789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:31.653708 25789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:31.653805 25789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:31.653900 25789 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:28:31.658749 25789 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:33915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:40835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:31.659960 25789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:31.661423 25789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:28:31.671339 25789 server_base.cc:1034] running on GCE node
W20260619 12:28:31.671267 25797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:31.671284 25795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:31.671267 25794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:31.672055 25789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:31.672775 25789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:31.673926 25789 hybrid_clock.cc:648] HybridClock initialized: now 1781872111673902 us; error 39 us; skew 500 ppm
I20260619 12:28:31.675863 25789 webserver.cc:460] Webserver started at http://127.6.231.193:37897/ using document root <none> and password file <none>
I20260619 12:28:31.676437 25789 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:31.676498 25789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:31.676741 25789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:31.678877 25789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "0d16d090eff74a29868528e34c52c928"
format_stamp: "Formatted at 2026-06-19 12:28:31 on dist-test-slave-kr9h"
server_key: "06d54fa55de2f3d08e32718d036a1160"
server_key_iv: "ef2eb016d3fe9b5c013248bce00d62e4"
server_key_version: "encryptionkey@0"
I20260619 12:28:31.679441 25789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "0d16d090eff74a29868528e34c52c928"
format_stamp: "Formatted at 2026-06-19 12:28:31 on dist-test-slave-kr9h"
server_key: "06d54fa55de2f3d08e32718d036a1160"
server_key_iv: "ef2eb016d3fe9b5c013248bce00d62e4"
server_key_version: "encryptionkey@0"
I20260619 12:28:31.683153 25789 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 12:28:31.685698 25803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:31.686975 25789 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:28:31.687134 25789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "0d16d090eff74a29868528e34c52c928"
format_stamp: "Formatted at 2026-06-19 12:28:31 on dist-test-slave-kr9h"
server_key: "06d54fa55de2f3d08e32718d036a1160"
server_key_iv: "ef2eb016d3fe9b5c013248bce00d62e4"
server_key_version: "encryptionkey@0"
I20260619 12:28:31.687263 25789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:31.717211 25789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:31.718035 25789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:31.718251 25789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:31.718883 25789 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:31.719866 25789 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:31.719938 25789 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:31.720011 25789 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:31.720072 25789 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:31.729225 25789 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:36947
I20260619 12:28:31.729262 25916 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:36947 every 8 connection(s)
I20260619 12:28:31.730237 25789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:28:31.732323 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 25789
I20260619 12:28:31.732458 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:28:31.732789 7071 external_mini_cluster.cc:1428] Setting key 2cff658f77c8d9faa4185ba729403b4a
I20260619 12:28:31.734947 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:40835
--builtin_ntp_servers=127.6.231.212:33915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:28:31.740372 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:31.732243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:55037 (local address 127.6.231.254:40835)
0619 12:28:31.732454 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:31.732460 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:31.734037 (+ 1577us) server_negotiation.cc:407] Connection header received
0619 12:28:31.735203 (+ 1166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:31.735207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:31.735261 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:31.735339 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:31.736370 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.736950 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.738306 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.738593 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.738730 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:31.739856 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:31.739859 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:31.739862 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:31.739875 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:31.739908 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:31.739917 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:31.739995 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:31.740164 (+ 169us) server_negotiation.cc:299] Negotiation successful
0619 12:28:31.740228 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260619 12:28:31.741259 25919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:31.732650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:40835 (local address 127.6.231.193:55037)
0619 12:28:31.733372 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:31.733493 (+ 121us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:31.734960 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:31.735519 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:31.735527 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:31.735614 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:31.736211 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:31.736231 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.737086 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.737093 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:31.738177 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:31.738186 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.738754 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.738758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:31.738951 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:31.739584 (+ 633us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:28:31.739586 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:31.739626 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:28:31.739693 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:31.740008 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:31.740019 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:31.740025 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:28:31.740164 (+ 139us) client_negotiation.cc:240] Negotiation successful
0619 12:28:31.740474 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":159,"threads_started":1}
I20260619 12:28:31.742589 25917 heartbeater.cc:347] Connected to a master server at 127.6.231.254:40835
I20260619 12:28:31.742866 25917 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:31.743615 25917 heartbeater.cc:511] Master 127.6.231.254:40835 requested a full tablet report, sending...
I20260619 12:28:31.745575 25729 ts_manager.cc:194] Registered new tserver with Master: 0d16d090eff74a29868528e34c52c928 (127.6.231.193:36947)
I20260619 12:28:31.747181 25729 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.231.193:55037
W20260619 12:28:31.852737 25920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:31.853030 25920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:31.853070 25920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:31.857075 25920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:31.857141 25920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:31.857204 25920 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:28:31.861972 25920 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:33915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:40835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:31.863061 25920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:31.864413 25920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:31.874164 25928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:31.874161 25926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:31.874305 25920 server_base.cc:1034] running on GCE node
W20260619 12:28:31.874154 25925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:31.874944 25920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:31.875679 25920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:31.876884 25920 hybrid_clock.cc:648] HybridClock initialized: now 1781872111876870 us; error 75 us; skew 500 ppm
I20260619 12:28:31.878971 25920 webserver.cc:460] Webserver started at http://127.6.231.194:38095/ using document root <none> and password file <none>
I20260619 12:28:31.879573 25920 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:31.879637 25920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:31.879884 25920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:31.882102 25920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "9fffbe567a344d3e9ac5a6a1a60519e7"
format_stamp: "Formatted at 2026-06-19 12:28:31 on dist-test-slave-kr9h"
server_key: "a23d7a82d3f0d44b9a321cee77a30d3a"
server_key_iv: "481c918f324c9a8e6d5b9c58639bae49"
server_key_version: "encryptionkey@0"
I20260619 12:28:31.882668 25920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "9fffbe567a344d3e9ac5a6a1a60519e7"
format_stamp: "Formatted at 2026-06-19 12:28:31 on dist-test-slave-kr9h"
server_key: "a23d7a82d3f0d44b9a321cee77a30d3a"
server_key_iv: "481c918f324c9a8e6d5b9c58639bae49"
server_key_version: "encryptionkey@0"
I20260619 12:28:31.886421 25920 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260619 12:28:31.888912 25934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:31.890242 25920 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:28:31.890391 25920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "9fffbe567a344d3e9ac5a6a1a60519e7"
format_stamp: "Formatted at 2026-06-19 12:28:31 on dist-test-slave-kr9h"
server_key: "a23d7a82d3f0d44b9a321cee77a30d3a"
server_key_iv: "481c918f324c9a8e6d5b9c58639bae49"
server_key_version: "encryptionkey@0"
I20260619 12:28:31.890817 25920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:31.908033 25920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:31.908813 25920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:31.909076 25920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:31.909793 25920 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:31.910778 25920 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:31.910852 25920 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:31.910925 25920 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:31.910969 25920 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:31.920210 25920 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:41303
I20260619 12:28:31.920228 26047 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:41303 every 8 connection(s)
I20260619 12:28:31.921167 25920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:28:31.921635 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 25920
I20260619 12:28:31.921767 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:28:31.922111 7071 external_mini_cluster.cc:1428] Setting key 881750a8f9dafe61b01836c45d892710
I20260619 12:28:31.924463 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:40835
--builtin_ntp_servers=127.6.231.212:33915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:28:31.932222 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:31.923210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:38499 (local address 127.6.231.254:40835)
0619 12:28:31.923358 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:31.923362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:31.924684 (+ 1322us) server_negotiation.cc:407] Connection header received
0619 12:28:31.926263 (+ 1579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:31.926270 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:31.926336 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:31.926443 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:31.927458 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.928380 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.929729 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.930167 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.930305 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:31.931585 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:31.931592 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:31.931597 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:31.931616 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:31.931645 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:31.931659 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:31.931746 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:31.931906 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 12:28:31.931988 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 12:28:31.933185 26050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:31.923726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:40835 (local address 127.6.231.194:38499)
0619 12:28:31.924495 (+ 769us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:31.924572 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:31.926044 (+ 1472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:31.926643 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:31.926650 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:31.926742 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:31.927285 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:31.927299 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.928532 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.928535 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:31.929604 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:31.929614 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:31.930347 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:31.930351 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:31.930530 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:31.931281 (+ 751us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:28:31.931283 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:31.931333 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:28:31.931406 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:31.931761 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:31.931768 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:31.931772 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:28:31.931873 (+ 101us) client_negotiation.cc:240] Negotiation successful
0619 12:28:31.932307 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":150,"threads_started":1}
I20260619 12:28:31.934602 26048 heartbeater.cc:347] Connected to a master server at 127.6.231.254:40835
I20260619 12:28:31.934911 26048 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:31.935604 26048 heartbeater.cc:511] Master 127.6.231.254:40835 requested a full tablet report, sending...
I20260619 12:28:31.936838 25729 ts_manager.cc:194] Registered new tserver with Master: 9fffbe567a344d3e9ac5a6a1a60519e7 (127.6.231.194:41303)
I20260619 12:28:31.937562 25729 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.231.194:38499
W20260619 12:28:32.041916 26051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:32.042217 26051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:32.042290 26051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:32.046344 26051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:32.046443 26051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:32.046547 26051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:28:32.051319 26051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:33915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:40835
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:32.052477 26051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:32.053925 26051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:32.063859 26057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:32.063884 26059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:32.063910 26051 server_base.cc:1034] running on GCE node
W20260619 12:28:32.063865 26056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:32.064534 26051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:32.065199 26051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:32.066366 26051 hybrid_clock.cc:648] HybridClock initialized: now 1781872112066326 us; error 57 us; skew 500 ppm
I20260619 12:28:32.068295 26051 webserver.cc:460] Webserver started at http://127.6.231.195:33017/ using document root <none> and password file <none>
I20260619 12:28:32.068885 26051 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:32.068948 26051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:32.069145 26051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:32.071388 26051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "afbc6a339d4b4e3e93adbb40250eee49"
format_stamp: "Formatted at 2026-06-19 12:28:32 on dist-test-slave-kr9h"
server_key: "2107cf30e4ac291bb7e4a510a02ecadd"
server_key_iv: "74eb77dfd5f09595ff3fce79e689596c"
server_key_version: "encryptionkey@0"
I20260619 12:28:32.071940 26051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "afbc6a339d4b4e3e93adbb40250eee49"
format_stamp: "Formatted at 2026-06-19 12:28:32 on dist-test-slave-kr9h"
server_key: "2107cf30e4ac291bb7e4a510a02ecadd"
server_key_iv: "74eb77dfd5f09595ff3fce79e689596c"
server_key_version: "encryptionkey@0"
I20260619 12:28:32.075690 26051 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 12:28:32.078337 26065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:32.079667 26051 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:32.079793 26051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "afbc6a339d4b4e3e93adbb40250eee49"
format_stamp: "Formatted at 2026-06-19 12:28:32 on dist-test-slave-kr9h"
server_key: "2107cf30e4ac291bb7e4a510a02ecadd"
server_key_iv: "74eb77dfd5f09595ff3fce79e689596c"
server_key_version: "encryptionkey@0"
I20260619 12:28:32.079908 26051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:32.096078 26051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:32.096812 26051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:32.097014 26051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:32.097795 26051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:32.098769 26051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:32.098842 26051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:32.098917 26051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:32.098963 26051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:32.107905 26051 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:37341
I20260619 12:28:32.107987 26178 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:37341 every 8 connection(s)
I20260619 12:28:32.108901 26051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:28:32.111385 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 26051
I20260619 12:28:32.111479 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:28:32.111802 7071 external_mini_cluster.cc:1428] Setting key 0b2de51ace8603319dce8f3a8a04e0f7
I20260619 12:28:32.118716 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.110952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:51357 (local address 127.6.231.254:40835)
0619 12:28:32.111116 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:32.111121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:32.112418 (+ 1297us) server_negotiation.cc:407] Connection header received
0619 12:28:32.113877 (+ 1459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:32.113881 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:32.113934 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:32.114011 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:32.114924 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.115458 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.116626 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.116870 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.116982 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.118202 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:32.118205 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:32.118208 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:32.118221 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:32.118247 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:32.118255 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:32.118331 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:32.118492 (+ 161us) server_negotiation.cc:299] Negotiation successful
0619 12:28:32.118568 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 12:28:32.119649 26181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.111463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:40835 (local address 127.6.231.195:51357)
0619 12:28:32.112240 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:32.112292 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:32.113670 (+ 1378us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:32.114190 (+ 520us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:32.114198 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:32.114287 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:32.114784 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.114797 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.115597 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.115605 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.116502 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.116510 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.117043 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.117048 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.117214 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.117918 (+ 704us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:28:32.117920 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:32.117961 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:28:32.118036 (+ 75us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:32.118343 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:32.118354 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:32.118360 (+ 6us) client_negotiation.cc:769] Sending connection context
0619 12:28:32.118492 (+ 132us) client_negotiation.cc:240] Negotiation successful
0619 12:28:32.118780 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":144,"threads_started":1}
I20260619 12:28:32.120816 26179 heartbeater.cc:347] Connected to a master server at 127.6.231.254:40835
I20260619 12:28:32.121099 26179 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:32.121666 26179 heartbeater.cc:511] Master 127.6.231.254:40835 requested a full tablet report, sending...
I20260619 12:28:32.122735 25729 ts_manager.cc:194] Registered new tserver with Master: afbc6a339d4b4e3e93adbb40250eee49 (127.6.231.195:37341)
I20260619 12:28:32.123322 25729 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.231.195:51357
I20260619 12:28:32.125998 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:28:32.131752 25768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.127975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32908 (local address 127.6.231.254:40835)
0619 12:28:32.128102 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:32.128105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:32.128253 (+ 148us) server_negotiation.cc:407] Connection header received
0619 12:28:32.128349 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:32.128351 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:32.128388 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:32.128454 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:32.128667 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.129163 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.130138 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.130344 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.130424 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.131284 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:32.131287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:32.131289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:32.131302 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:32.131322 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:32.131330 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:32.131450 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:32.131601 (+ 151us) server_negotiation.cc:299] Negotiation successful
0619 12:28:32.131635 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260619 12:28:32.135030 25729 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:32908:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:28:32.138218 25729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:28:32.153594 26194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.148363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:41303 (local address 127.0.0.1:51210)
0619 12:28:32.149120 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:32.149133 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:32.149211 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:32.149884 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:32.149887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:32.149906 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:32.150004 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.150010 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.151207 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.151210 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.152262 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.152272 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.153123 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.153127 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.153228 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.153251 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 12:28:32.153315 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 12:28:32.153387 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":32,"thread_start_us":82,"threads_started":1}
I20260619 12:28:32.153631 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.148747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51210 (local address 127.6.231.194:41303)
0619 12:28:32.149459 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:32.149471 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:32.149494 (+ 23us) server_negotiation.cc:407] Connection header received
0619 12:28:32.149565 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:32.149570 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:32.149740 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:32.149883 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:32.150143 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.151074 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.152387 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.152983 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.153101 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.153291 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:32.153344 (+ 53us) server_negotiation.cc:299] Negotiation successful
0619 12:28:32.153470 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":581,"thread_start_us":101,"threads_started":1}
I20260619 12:28:32.153771 26191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.147290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51220 (local address 127.6.231.193:36947)
0619 12:28:32.148440 (+ 1150us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:32.148454 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:32.148478 (+ 24us) server_negotiation.cc:407] Connection header received
0619 12:28:32.148568 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:32.148573 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:32.148754 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:32.148912 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:32.149342 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.150278 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.152128 (+ 1850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.152782 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.152869 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.153028 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:32.153502 (+ 474us) server_negotiation.cc:299] Negotiation successful
0619 12:28:32.153601 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1035,"thread_start_us":94,"threads_started":1}
I20260619 12:28:32.154193 26190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.147123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:36947 (local address 127.0.0.1:51220)
0619 12:28:32.147784 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:32.147813 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:32.147957 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:32.149065 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:32.149069 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:32.149088 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:32.149216 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.149221 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.150379 (+ 1158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.150385 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.152016 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.152022 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.152914 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.152918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.153047 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.153105 (+ 58us) client_negotiation.cc:769] Sending connection context
0619 12:28:32.153211 (+ 106us) client_negotiation.cc:240] Negotiation successful
0619 12:28:32.153340 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":118,"spinlock_wait_cycles":5760,"thread_start_us":104,"threads_started":1}
I20260619 12:28:32.154562 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.147319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:37341 (local address 127.0.0.1:50816)
0619 12:28:32.147759 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:32.147813 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:32.148006 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:32.148935 (+ 929us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:32.148966 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:32.148994 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:32.149135 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.149141 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.150589 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.150592 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.151997 (+ 1405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.152008 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.154118 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.154122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.154225 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.154242 (+ 17us) client_negotiation.cc:769] Sending connection context
0619 12:28:32.154329 (+ 87us) client_negotiation.cc:240] Negotiation successful
0619 12:28:32.154403 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":77,"spinlock_wait_cycles":3072,"thread_start_us":140,"threads_started":1}
I20260619 12:28:32.154793 26193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.147658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50816 (local address 127.6.231.195:37341)
0619 12:28:32.148016 (+ 358us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:32.148028 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:32.148055 (+ 27us) server_negotiation.cc:407] Connection header received
0619 12:28:32.148363 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:32.148369 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:32.148592 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:32.148758 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:32.149256 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.150469 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.153112 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.153969 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.154091 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.154445 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:32.154503 (+ 58us) server_negotiation.cc:299] Negotiation successful
0619 12:28:32.154623 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":163,"threads_started":1}
I20260619 12:28:32.157491 26113 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1d8158d8ed4513b2e9d29814a1bb93 (DEFAULT_TABLE table=test-table [id=1e2d7107292941a797b5eb7179fc7333]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:32.157594 25982 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1d8158d8ed4513b2e9d29814a1bb93 (DEFAULT_TABLE table=test-table [id=1e2d7107292941a797b5eb7179fc7333]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:32.157593 25851 tablet_service.cc:1467] Processing CreateTablet for tablet 0e1d8158d8ed4513b2e9d29814a1bb93 (DEFAULT_TABLE table=test-table [id=1e2d7107292941a797b5eb7179fc7333]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:32.158598 26113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1d8158d8ed4513b2e9d29814a1bb93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:32.158644 25851 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1d8158d8ed4513b2e9d29814a1bb93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:32.158644 25982 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e1d8158d8ed4513b2e9d29814a1bb93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:32.165159 26196 tablet_bootstrap.cc:492] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49: Bootstrap starting.
I20260619 12:28:32.166661 26197 tablet_bootstrap.cc:492] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928: Bootstrap starting.
I20260619 12:28:32.167563 26196 tablet_bootstrap.cc:654] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:32.168305 26196 log.cc:824] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:32.168540 26197 tablet_bootstrap.cc:654] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:32.169005 26198 tablet_bootstrap.cc:492] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7: Bootstrap starting.
I20260619 12:28:32.169377 26197 log.cc:824] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:32.170553 26196 tablet_bootstrap.cc:492] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49: No bootstrap required, opened a new log
I20260619 12:28:32.170748 26196 ts_tablet_manager.cc:1397] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 12:28:32.171186 26198 tablet_bootstrap.cc:654] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:32.171480 26197 tablet_bootstrap.cc:492] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928: No bootstrap required, opened a new log
I20260619 12:28:32.171679 26197 ts_tablet_manager.cc:1397] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.003s
I20260619 12:28:32.172004 26198 log.cc:824] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:32.173795 26198 tablet_bootstrap.cc:492] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7: No bootstrap required, opened a new log
I20260619 12:28:32.173990 26198 ts_tablet_manager.cc:1397] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 12:28:32.175305 26196 raft_consensus.cc:348] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d16d090eff74a29868528e34c52c928" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36947 } } peers { permanent_uuid: "afbc6a339d4b4e3e93adbb40250eee49" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37341 } } peers { permanent_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41303 } }
I20260619 12:28:32.175598 26196 raft_consensus.cc:374] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:32.175660 26196 raft_consensus.cc:729] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afbc6a339d4b4e3e93adbb40250eee49, State: Initialized, Role: FOLLOWER
I20260619 12:28:32.175863 26197 raft_consensus.cc:348] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d16d090eff74a29868528e34c52c928" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36947 } } peers { permanent_uuid: "afbc6a339d4b4e3e93adbb40250eee49" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37341 } } peers { permanent_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41303 } }
I20260619 12:28:32.176110 26197 raft_consensus.cc:374] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:32.176162 26197 raft_consensus.cc:729] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d16d090eff74a29868528e34c52c928, State: Initialized, Role: FOLLOWER
I20260619 12:28:32.176151 26196 consensus_queue.cc:260] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d16d090eff74a29868528e34c52c928" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36947 } } peers { permanent_uuid: "afbc6a339d4b4e3e93adbb40250eee49" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37341 } } peers { permanent_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41303 } }
I20260619 12:28:32.176647 26197 consensus_queue.cc:260] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d16d090eff74a29868528e34c52c928" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36947 } } peers { permanent_uuid: "afbc6a339d4b4e3e93adbb40250eee49" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37341 } } peers { permanent_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41303 } }
I20260619 12:28:32.177249 26196 ts_tablet_manager.cc:1428] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 12:28:32.177598 26197 ts_tablet_manager.cc:1428] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260619 12:28:32.177938 25917 heartbeater.cc:503] Master 127.6.231.254:40835 was elected leader, sending a full tablet report...
I20260619 12:28:32.177805 26198 raft_consensus.cc:348] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d16d090eff74a29868528e34c52c928" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36947 } } peers { permanent_uuid: "afbc6a339d4b4e3e93adbb40250eee49" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37341 } } peers { permanent_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41303 } }
I20260619 12:28:32.178050 26198 raft_consensus.cc:374] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:32.178117 26198 raft_consensus.cc:729] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fffbe567a344d3e9ac5a6a1a60519e7, State: Initialized, Role: FOLLOWER
I20260619 12:28:32.178386 26179 heartbeater.cc:503] Master 127.6.231.254:40835 was elected leader, sending a full tablet report...
I20260619 12:28:32.178614 26198 consensus_queue.cc:260] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d16d090eff74a29868528e34c52c928" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36947 } } peers { permanent_uuid: "afbc6a339d4b4e3e93adbb40250eee49" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37341 } } peers { permanent_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41303 } }
I20260619 12:28:32.179267 26048 heartbeater.cc:503] Master 127.6.231.254:40835 was elected leader, sending a full tablet report...
I20260619 12:28:32.181063 26198 ts_tablet_manager.cc:1428] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260619 12:28:32.231551 25918 tablet.cc:2369] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:28:32.360060 26180 tablet.cc:2369] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260619 12:28:32.422402 26049 tablet.cc:2369] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:28:32.557701 26202 raft_consensus.cc:482] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:28:32.557967 26202 raft_consensus.cc:504] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d16d090eff74a29868528e34c52c928" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36947 } } peers { permanent_uuid: "afbc6a339d4b4e3e93adbb40250eee49" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37341 } } peers { permanent_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41303 } }
I20260619 12:28:32.559505 26202 leader_election.cc:288] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0d16d090eff74a29868528e34c52c928 (127.6.231.193:36947), 9fffbe567a344d3e9ac5a6a1a60519e7 (127.6.231.194:41303)
I20260619 12:28:32.563685 26181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.559743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:36947 (local address 127.6.231.195:49745)
0619 12:28:32.559883 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:32.559900 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:32.560007 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:32.560401 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:32.560405 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:32.560425 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:32.560553 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.560564 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.561481 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.561485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.562472 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.562482 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.563247 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.563251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.563357 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.563398 (+ 41us) client_negotiation.cc:769] Sending connection context
0619 12:28:32.563475 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 12:28:32.563554 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56,"mutex_wait_us":1}
I20260619 12:28:32.563755 26191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.559898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:49745 (local address 127.6.231.193:36947)
0619 12:28:32.560067 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:32.560072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:32.560089 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:28:32.560159 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:32.560162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:32.560212 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:32.560328 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:32.560702 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.561361 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.562614 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.563111 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.563205 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.563244 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:32.563525 (+ 281us) server_negotiation.cc:299] Negotiation successful
0619 12:28:32.563607 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260619 12:28:32.563970 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.560232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:53481 (local address 127.6.231.194:41303)
0619 12:28:32.560368 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:32.560373 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:32.560576 (+ 203us) server_negotiation.cc:407] Connection header received
0619 12:28:32.560640 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:32.560644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:32.560703 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:32.560760 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:32.561062 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.561681 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.562854 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.563327 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.563484 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.563527 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:32.563762 (+ 235us) server_negotiation.cc:299] Negotiation successful
0619 12:28:32.563839 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260619 12:28:32.563977 26205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.560078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:41303 (local address 127.6.231.195:53481)
0619 12:28:32.560477 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:32.560489 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:32.560570 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:32.560796 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:32.560799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:32.560815 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:32.560934 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.560940 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.561810 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.561813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.562719 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.562729 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.563517 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.563521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.563624 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.563646 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 12:28:32.563722 (+ 76us) client_negotiation.cc:240] Negotiation successful
0619 12:28:32.563787 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":313,"mutex_wait_us":1,"thread_start_us":137,"threads_started":1}
I20260619 12:28:32.564397 25871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d8158d8ed4513b2e9d29814a1bb93" candidate_uuid: "afbc6a339d4b4e3e93adbb40250eee49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d16d090eff74a29868528e34c52c928" is_pre_election: true
I20260619 12:28:32.564668 26203 raft_consensus.cc:482] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:28:32.564867 26203 raft_consensus.cc:504] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d16d090eff74a29868528e34c52c928" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36947 } } peers { permanent_uuid: "afbc6a339d4b4e3e93adbb40250eee49" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37341 } } peers { permanent_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41303 } }
I20260619 12:28:32.565192 26002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d8158d8ed4513b2e9d29814a1bb93" candidate_uuid: "afbc6a339d4b4e3e93adbb40250eee49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" is_pre_election: true
I20260619 12:28:32.565327 25871 raft_consensus.cc:2454] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afbc6a339d4b4e3e93adbb40250eee49 in term 0.
I20260619 12:28:32.565521 26002 raft_consensus.cc:2454] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate afbc6a339d4b4e3e93adbb40250eee49 in term 0.
I20260619 12:28:32.565856 26068 leader_election.cc:302] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d16d090eff74a29868528e34c52c928, afbc6a339d4b4e3e93adbb40250eee49; no voters:
I20260619 12:28:32.566160 26203 leader_election.cc:288] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers afbc6a339d4b4e3e93adbb40250eee49 (127.6.231.195:37341), 9fffbe567a344d3e9ac5a6a1a60519e7 (127.6.231.194:41303)
I20260619 12:28:32.566169 26202 raft_consensus.cc:2781] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:28:32.566267 26202 raft_consensus.cc:482] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:28:32.566314 26202 raft_consensus.cc:3037] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:32.567803 26202 raft_consensus.cc:504] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d16d090eff74a29868528e34c52c928" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36947 } } peers { permanent_uuid: "afbc6a339d4b4e3e93adbb40250eee49" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37341 } } peers { permanent_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41303 } }
I20260619 12:28:32.568231 26202 leader_election.cc:288] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [CANDIDATE]: Term 1 election: Requested vote from peers 0d16d090eff74a29868528e34c52c928 (127.6.231.193:36947), 9fffbe567a344d3e9ac5a6a1a60519e7 (127.6.231.194:41303)
I20260619 12:28:32.568617 26002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d8158d8ed4513b2e9d29814a1bb93" candidate_uuid: "afbc6a339d4b4e3e93adbb40250eee49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7"
I20260619 12:28:32.568753 26002 raft_consensus.cc:3037] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:32.570168 26002 raft_consensus.cc:2454] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afbc6a339d4b4e3e93adbb40250eee49 in term 1.
I20260619 12:28:32.570540 26195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.566521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:55543 (local address 127.6.231.194:41303)
0619 12:28:32.566627 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:32.566631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:32.567104 (+ 473us) server_negotiation.cc:407] Connection header received
0619 12:28:32.567185 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:32.567187 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:32.567230 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:32.567314 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:32.567586 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.568128 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.569542 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.569938 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.570073 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.570123 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:32.570349 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 12:28:32.570413 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 12:28:32.570549 25871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d8158d8ed4513b2e9d29814a1bb93" candidate_uuid: "afbc6a339d4b4e3e93adbb40250eee49" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0d16d090eff74a29868528e34c52c928"
I20260619 12:28:32.570561 26207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.566472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:41303 (local address 127.6.231.193:55543)
0619 12:28:32.567000 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:32.567013 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:32.567106 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:32.567336 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:32.567339 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:32.567351 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:32.567474 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.567480 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.568269 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.568276 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.569396 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.569406 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.570073 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.570077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.570172 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.570195 (+ 23us) client_negotiation.cc:769] Sending connection context
0619 12:28:32.570332 (+ 137us) client_negotiation.cc:240] Negotiation successful
0619 12:28:32.570409 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":49,"thread_start_us":71,"threads_started":1}
I20260619 12:28:32.570724 25871 raft_consensus.cc:3037] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:32.571065 26193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.566707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:56369 (local address 127.6.231.195:37341)
0619 12:28:32.566840 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:32.566843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:32.566855 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:28:32.566977 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:32.566980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:32.567032 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:32.567105 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:32.567448 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.568148 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.569750 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.570331 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.570427 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.570477 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:32.570844 (+ 367us) server_negotiation.cc:299] Negotiation successful
0619 12:28:32.570929 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260619 12:28:32.571394 26206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.566332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:37341 (local address 127.6.231.193:56369)
0619 12:28:32.566764 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:32.566779 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:32.566861 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:32.567209 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:32.567212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:32.567234 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:32.567353 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.567359 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.568286 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.568289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.569608 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:32.569621 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.570583 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.570586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:32.570683 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.570705 (+ 22us) client_negotiation.cc:769] Sending connection context
0619 12:28:32.571195 (+ 490us) client_negotiation.cc:240] Negotiation successful
0619 12:28:32.571270 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":100,"thread_start_us":73,"threads_started":1}
I20260619 12:28:32.572137 25871 raft_consensus.cc:2454] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate afbc6a339d4b4e3e93adbb40250eee49 in term 1.
I20260619 12:28:32.572428 26002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d8158d8ed4513b2e9d29814a1bb93" candidate_uuid: "0d16d090eff74a29868528e34c52c928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" is_pre_election: true
I20260619 12:28:32.572619 26002 raft_consensus.cc:2379] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 0d16d090eff74a29868528e34c52c928 in current term 1: Already voted for candidate afbc6a339d4b4e3e93adbb40250eee49 in this term.
I20260619 12:28:32.572965 26068 leader_election.cc:302] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0d16d090eff74a29868528e34c52c928, afbc6a339d4b4e3e93adbb40250eee49; no voters:
I20260619 12:28:32.573284 26202 raft_consensus.cc:2781] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:32.573572 26202 raft_consensus.cc:686] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [term 1 LEADER]: Becoming Leader. State: Replica: afbc6a339d4b4e3e93adbb40250eee49, State: Running, Role: LEADER
I20260619 12:28:32.573710 26133 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e1d8158d8ed4513b2e9d29814a1bb93" candidate_uuid: "0d16d090eff74a29868528e34c52c928" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afbc6a339d4b4e3e93adbb40250eee49" is_pre_election: true
I20260619 12:28:32.573943 26202 consensus_queue.cc:237] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d16d090eff74a29868528e34c52c928" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36947 } } peers { permanent_uuid: "afbc6a339d4b4e3e93adbb40250eee49" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37341 } } peers { permanent_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41303 } }
I20260619 12:28:32.574316 25805 leader_election.cc:302] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0d16d090eff74a29868528e34c52c928; no voters: 9fffbe567a344d3e9ac5a6a1a60519e7, afbc6a339d4b4e3e93adbb40250eee49
I20260619 12:28:32.574638 26203 raft_consensus.cc:2726] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260619 12:28:32.577387 25729 catalog_manager.cc:5495] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 reported cstate change: term changed from 0 to 1, leader changed from <none> to afbc6a339d4b4e3e93adbb40250eee49 (127.6.231.195). New cstate: current_term: 1 leader_uuid: "afbc6a339d4b4e3e93adbb40250eee49" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0d16d090eff74a29868528e34c52c928" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36947 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afbc6a339d4b4e3e93adbb40250eee49" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 37341 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41303 } health_report { overall_health: UNKNOWN } } }
I20260619 12:28:32.609579 26193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.605705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50820 (local address 127.6.231.195:37341)
0619 12:28:32.605867 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:32.605872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:32.605887 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:28:32.606071 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:32.606073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:32.606129 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:32.606244 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:28:32.606520 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.607049 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.607937 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:32.608159 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:32.608272 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:32.608604 (+ 332us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:28:32.609047 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:28:32.609208 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:32.609294 (+ 86us) server_negotiation.cc:299] Negotiation successful
0619 12:28:32.609328 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 12:28:32.615743 26002 raft_consensus.cc:1261] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7 [term 1 FOLLOWER]: Refusing update from remote peer afbc6a339d4b4e3e93adbb40250eee49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:28:32.615787 25871 raft_consensus.cc:1261] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 0d16d090eff74a29868528e34c52c928 [term 1 FOLLOWER]: Refusing update from remote peer afbc6a339d4b4e3e93adbb40250eee49: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:28:32.616489 26208 consensus_queue.cc:1035] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9fffbe567a344d3e9ac5a6a1a60519e7" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 41303 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:28:32.616739 26202 consensus_queue.cc:1035] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0d16d090eff74a29868528e34c52c928" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 36947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:28:32.623867 26212 mvcc.cc:205] Tried to move back new op lower bound from 7298548173267013632 to 7298548173105618944. Current Snapshot: MvccSnapshot[applied={T|T < 7298548173267013632}]
I20260619 12:28:32.624560 26211 mvcc.cc:205] Tried to move back new op lower bound from 7298548173267013632 to 7298548173105618944. Current Snapshot: MvccSnapshot[applied={T|T < 7298548173267013632}]
I20260619 12:28:32.632985 25729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:32908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:28:32.633185 25729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:32908:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:28:32.635795 25729 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c46f73338e524f3b9913084278c26d16: Sending DeleteTablet for 3 replicas of tablet 0e1d8158d8ed4513b2e9d29814a1bb93
I20260619 12:28:32.636642 26113 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1d8158d8ed4513b2e9d29814a1bb93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:32 UTC) from {username='slave'} at 127.0.0.1:50816
I20260619 12:28:32.636813 25851 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1d8158d8ed4513b2e9d29814a1bb93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:32 UTC) from {username='slave'} at 127.0.0.1:51220
I20260619 12:28:32.637280 25982 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e1d8158d8ed4513b2e9d29814a1bb93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:32 UTC) from {username='slave'} at 127.0.0.1:51210
I20260619 12:28:32.637645 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 25789
I20260619 12:28:32.638875 26222 tablet_replica.cc:330] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49: stopping tablet replica
I20260619 12:28:32.639240 26222 raft_consensus.cc:2229] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:28:32.639750 26222 raft_consensus.cc:2258] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:28:32.640278 26221 tablet_replica.cc:330] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7: stopping tablet replica
I20260619 12:28:32.640599 26221 raft_consensus.cc:2229] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:28:32.640940 26221 raft_consensus.cc:2258] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:28:32.641418 26222 ts_tablet_manager.cc:1905] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:28:32.642841 26221 ts_tablet_manager.cc:1905] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:28:32.645078 26222 ts_tablet_manager.cc:1918] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:28:32.645198 26222 log.cc:1196] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/0e1d8158d8ed4513b2e9d29814a1bb93
I20260619 12:28:32.645555 26222 ts_tablet_manager.cc:1939] T 0e1d8158d8ed4513b2e9d29814a1bb93 P afbc6a339d4b4e3e93adbb40250eee49: Deleting consensus metadata
I20260619 12:28:32.645615 26221 ts_tablet_manager.cc:1918] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:28:32.645725 26221 log.cc:1196] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/0e1d8158d8ed4513b2e9d29814a1bb93
I20260619 12:28:32.646034 26221 ts_tablet_manager.cc:1939] T 0e1d8158d8ed4513b2e9d29814a1bb93 P 9fffbe567a344d3e9ac5a6a1a60519e7: Deleting consensus metadata
I20260619 12:28:32.646999 25714 catalog_manager.cc:4841] TS afbc6a339d4b4e3e93adbb40250eee49 (127.6.231.195:37341): tablet 0e1d8158d8ed4513b2e9d29814a1bb93 (table test-table [id=1e2d7107292941a797b5eb7179fc7333]) successfully deleted
I20260619 12:28:32.647354 25715 catalog_manager.cc:4841] TS 9fffbe567a344d3e9ac5a6a1a60519e7 (127.6.231.194:41303): tablet 0e1d8158d8ed4513b2e9d29814a1bb93 (table test-table [id=1e2d7107292941a797b5eb7179fc7333]) successfully deleted
W20260619 12:28:32.647646 25715 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:36947 (error 108)
I20260619 12:28:32.647856 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 25920
I20260619 12:28:32.654547 26194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:32.654158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:36947 (local address 127.0.0.1:51236)
0619 12:28:32.654317 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:32.654440 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:36947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260619 12:28:32.654817 25715 catalog_manager.cc:4584] TS 0d16d090eff74a29868528e34c52c928 (127.6.231.193:36947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e1d8158d8ed4513b2e9d29814a1bb93: Network error: Client connection negotiation failed: client connection to 127.6.231.193:36947: connect: Connection refused (error 111)
I20260619 12:28:32.655166 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 26051
I20260619 12:28:32.660837 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 25698
2026-06-19T12:28:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1409 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:32 dist-test-slave-kr9h krb5kdc[26226](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:32 dist-test-slave-kr9h krb5kdc[26226](info): set up 2 sockets
Jun 19 12:28:32 dist-test-slave-kr9h krb5kdc[26226](info): commencing operation
krb5kdc: starting...
W20260619 12:28:33.721822 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:33 dist-test-slave-kr9h krb5kdc[26226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:28:33.854056 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:35033
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:35033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:33.974179 26239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:33.974515 26239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:33.974596 26239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:33.978858 26239 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:33.978963 26239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:33.979000 26239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:33.979038 26239 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:33.979082 26239 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:33.988193 26239 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:35033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:35033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:33.990097 26239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:33.991703 26239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:34.003563 26247 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:34.003563 26245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:34.003584 26244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:34.004160 26239 server_base.cc:1034] running on GCE node
I20260619 12:28:34.004817 26239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:34.006137 26239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:34.007386 26239 hybrid_clock.cc:648] HybridClock initialized: now 1781872114007346 us; error 59 us; skew 500 ppm
Jun 19 12:28:34 dist-test-slave-kr9h krb5kdc[26226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872114, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:34.011574 26239 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:28:34.012862 26239 webserver.cc:460] Webserver started at http://127.6.231.254:37147/ using document root <none> and password file <none>
I20260619 12:28:34.013752 26239 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:34.013854 26239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:34.014118 26239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:34.017211 26239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "ac449efd133b457dafdf687d807d7d4f"
format_stamp: "Formatted at 2026-06-19 12:28:34 on dist-test-slave-kr9h"
server_key: "fd35773015dd7e5ca8248c258ec82fa4"
server_key_iv: "ddea4cbee285bbdfb51314e43805d215"
server_key_version: "encryptionkey@0"
I20260619 12:28:34.018057 26239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "ac449efd133b457dafdf687d807d7d4f"
format_stamp: "Formatted at 2026-06-19 12:28:34 on dist-test-slave-kr9h"
server_key: "fd35773015dd7e5ca8248c258ec82fa4"
server_key_iv: "ddea4cbee285bbdfb51314e43805d215"
server_key_version: "encryptionkey@0"
I20260619 12:28:34.023684 26239 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 12:28:34.027523 26254 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:34.029289 26239 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:28:34.029480 26239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "ac449efd133b457dafdf687d807d7d4f"
format_stamp: "Formatted at 2026-06-19 12:28:34 on dist-test-slave-kr9h"
server_key: "fd35773015dd7e5ca8248c258ec82fa4"
server_key_iv: "ddea4cbee285bbdfb51314e43805d215"
server_key_version: "encryptionkey@0"
I20260619 12:28:34.029740 26239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:34.052650 26239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:34.056178 26239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:34.056386 26239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:34.064055 26239 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:35033
I20260619 12:28:34.064074 26306 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:35033 every 8 connection(s)
I20260619 12:28:34.065097 26239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:34.068055 26307 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:34.071756 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 26239
I20260619 12:28:34.071905 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:34.072211 7071 external_mini_cluster.cc:1428] Setting key d71f5d1a3ff75476820ea60fa4e2058e
I20260619 12:28:34.074550 26307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f: Bootstrap starting.
I20260619 12:28:34.076920 26307 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:34.077790 26307 log.cc:824] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:34.080076 26307 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f: No bootstrap required, opened a new log
Jun 19 12:28:34 dist-test-slave-kr9h krb5kdc[26226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:28:34.084295 26307 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac449efd133b457dafdf687d807d7d4f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35033 } }
I20260619 12:28:34.084580 26307 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:34.084666 26307 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac449efd133b457dafdf687d807d7d4f, State: Initialized, Role: FOLLOWER
I20260619 12:28:34.085208 26307 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac449efd133b457dafdf687d807d7d4f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35033 } }
I20260619 12:28:34.085366 26307 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:34.085462 26307 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:34.085569 26307 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:34.086611 26307 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac449efd133b457dafdf687d807d7d4f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35033 } }
I20260619 12:28:34.087003 26307 leader_election.cc:302] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac449efd133b457dafdf687d807d7d4f; no voters:
I20260619 12:28:34.087404 26307 leader_election.cc:288] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:34.087483 26312 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:34.087658 26312 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [term 1 LEADER]: Becoming Leader. State: Replica: ac449efd133b457dafdf687d807d7d4f, State: Running, Role: LEADER
I20260619 12:28:34.087738 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.073867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60858 (local address 127.6.231.254:35033)
0619 12:28:34.074389 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:34.074404 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:34.074442 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:28:34.075150 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:34.075170 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:34.075535 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:34.075896 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:34.076746 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.077921 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.079039 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.079543 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.079720 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.082374 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:34.082380 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:34.082395 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:34.082423 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:34.084401 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:34.084947 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:34.084952 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:34.084957 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:34.085080 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:34.085375 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:34.085379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:34.085381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:34.085922 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:34.086144 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:34.086427 (+ 283us) server_negotiation.cc:299] Negotiation successful
0619 12:28:34.086775 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":149,"threads_started":1}
I20260619 12:28:34.087957 26312 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac449efd133b457dafdf687d807d7d4f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35033 } }
I20260619 12:28:34.088730 26307 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:34.090519 26314 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac449efd133b457dafdf687d807d7d4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac449efd133b457dafdf687d807d7d4f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35033 } } }
I20260619 12:28:34.090780 26314 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:34.091012 26313 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac449efd133b457dafdf687d807d7d4f. Latest consensus state: current_term: 1 leader_uuid: "ac449efd133b457dafdf687d807d7d4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac449efd133b457dafdf687d807d7d4f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 35033 } } }
I20260619 12:28:34.091512 26313 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f [sys.catalog]: This master's current role is: LEADER
W20260619 12:28:34.093385 26327 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:28:34.093520 26327 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:28:34.093629 26328 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:34.094350 26328 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:34.100471 26328 catalog_manager.cc:1347] Generated new cluster ID: 446902e75fdc4fffac79873a32c78d9e
I20260619 12:28:34.100782 26328 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:34.116164 26328 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:34.117547 26328 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:34.138031 26328 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f: Generated new TSK 0
I20260619 12:28:34.138983 26328 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.6.231.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
Entry for principal kudu/127.6.231.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab.
I20260619 12:28:34.173892 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35033
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:34.295024 26333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:34.295394 26333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:34.295486 26333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:34.299953 26333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:34.300082 26333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:34.300185 26333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:28:34.306064 26333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.193.keytab
--principal=kudu/127.6.231.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:34.307325 26333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:34.308715 26333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:34.319546 26338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:34.319546 26339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:34.319630 26333 server_base.cc:1034] running on GCE node
W20260619 12:28:34.319545 26341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:34.320433 26333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:34.321241 26333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:34.322474 26333 hybrid_clock.cc:648] HybridClock initialized: now 1781872114322458 us; error 41 us; skew 500 ppm
Jun 19 12:28:34 dist-test-slave-kr9h krb5kdc[26226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872114, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:34.325779 26333 init.cc:377] Logged in from keytab as kudu/127.6.231.193@KRBTEST.COM (short username kudu)
I20260619 12:28:34.326853 26333 webserver.cc:460] Webserver started at http://127.6.231.193:41995/ using document root <none> and password file <none>
I20260619 12:28:34.327481 26333 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:34.327581 26333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:34.327821 26333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:34.330071 26333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "1b34602eff2a47358ea3d16afccb5f4a"
format_stamp: "Formatted at 2026-06-19 12:28:34 on dist-test-slave-kr9h"
server_key: "3e304cf05cb555d1e5e02a2240823684"
server_key_iv: "3b53716221620a5045a1d082028d1bf7"
server_key_version: "encryptionkey@0"
I20260619 12:28:34.330677 26333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "1b34602eff2a47358ea3d16afccb5f4a"
format_stamp: "Formatted at 2026-06-19 12:28:34 on dist-test-slave-kr9h"
server_key: "3e304cf05cb555d1e5e02a2240823684"
server_key_iv: "3b53716221620a5045a1d082028d1bf7"
server_key_version: "encryptionkey@0"
I20260619 12:28:34.334893 26333 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:28:34.337837 26348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:34.339350 26333 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 12:28:34.339540 26333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "1b34602eff2a47358ea3d16afccb5f4a"
format_stamp: "Formatted at 2026-06-19 12:28:34 on dist-test-slave-kr9h"
server_key: "3e304cf05cb555d1e5e02a2240823684"
server_key_iv: "3b53716221620a5045a1d082028d1bf7"
server_key_version: "encryptionkey@0"
I20260619 12:28:34.339681 26333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:34.356482 26333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:34.360448 26333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:34.360797 26333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:34.361677 26333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:34.362761 26333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:34.362840 26333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:34.362918 26333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:34.362969 26333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:34.375615 26333 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:35751
I20260619 12:28:34.375639 26461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:35751 every 8 connection(s)
I20260619 12:28:34.376780 26333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:28:34.381577 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 26333
I20260619 12:28:34.381739 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:28:34.382058 7071 external_mini_cluster.cc:1428] Setting key 141a66da769f7ffbcfca00086aa81cae
Jun 19 12:28:34 dist-test-slave-kr9h krb5kdc[26226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872114, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.193@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.194@KRBTEST.COM; defaulting to no policy
I20260619 12:28:34.395574 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.379488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:54927 (local address 127.6.231.254:35033)
0619 12:28:34.379740 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:34.379745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:34.381121 (+ 1376us) server_negotiation.cc:407] Connection header received
0619 12:28:34.382348 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:34.382352 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:34.382425 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:34.382599 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:34.384253 (+ 1654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.385261 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.386829 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.387096 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.387229 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.390510 (+ 3281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:34.390517 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:34.390522 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:34.390562 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:34.392661 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:34.393469 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:34.393473 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:34.393475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:34.393575 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:34.394006 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:34.394009 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:34.394010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:34.394343 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:34.394431 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:34.395206 (+ 775us) server_negotiation.cc:299] Negotiation successful
0619 12:28:34.395399 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.194@KRBTEST.COM" created.
I20260619 12:28:34.396504 26464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.380012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35033 (local address 127.6.231.193:54927)
0619 12:28:34.380885 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:34.380939 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:34.382043 (+ 1104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:34.382880 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:34.382889 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:34.383353 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:34.384065 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.384082 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.385503 (+ 1421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.385507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.386681 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.386693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.387289 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.387294 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.387505 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.388377 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:28:34.388397 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:34.390278 (+ 1881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:34.392841 (+ 2563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:34.392848 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:34.392864 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:34.393297 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:34.393690 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:34.393693 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:34.393695 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:34.393886 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:34.394475 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:34.394482 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:34.394837 (+ 355us) client_negotiation.cc:769] Sending connection context
0619 12:28:34.395035 (+ 198us) client_negotiation.cc:240] Negotiation successful
0619 12:28:34.395417 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":626,"thread_start_us":139,"threads_started":1}
I20260619 12:28:34.398244 26462 heartbeater.cc:347] Connected to a master server at 127.6.231.254:35033
I20260619 12:28:34.398576 26462 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:34.399163 26462 heartbeater.cc:511] Master 127.6.231.254:35033 requested a full tablet report, sending...
I20260619 12:28:34.401052 26271 ts_manager.cc:194] Registered new tserver with Master: 1b34602eff2a47358ea3d16afccb5f4a (127.6.231.193:35751)
I20260619 12:28:34.402462 26271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.193@KRBTEST.COM'} at 127.6.231.193:54927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
Entry for principal kudu/127.6.231.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab.
I20260619 12:28:34.416649 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35033
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:34.540933 26467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:34.541311 26467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:34.541400 26467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:34.546183 26467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:34.546289 26467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:34.546388 26467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:28:34.552809 26467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.194.keytab
--principal=kudu/127.6.231.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:34.554349 26467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:34.555608 26467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:34.566847 26473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:34.566849 26472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:34.566849 26475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:34.567392 26467 server_base.cc:1034] running on GCE node
I20260619 12:28:34.567886 26467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:34.568656 26467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:34.569860 26467 hybrid_clock.cc:648] HybridClock initialized: now 1781872114569804 us; error 74 us; skew 500 ppm
Jun 19 12:28:34 dist-test-slave-kr9h krb5kdc[26226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872114, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:34.573007 26467 init.cc:377] Logged in from keytab as kudu/127.6.231.194@KRBTEST.COM (short username kudu)
I20260619 12:28:34.574110 26467 webserver.cc:460] Webserver started at http://127.6.231.194:38515/ using document root <none> and password file <none>
I20260619 12:28:34.574690 26467 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:34.574750 26467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:34.574923 26467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:34.577052 26467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "f8b304332541468aa39ec45ead315e8a"
format_stamp: "Formatted at 2026-06-19 12:28:34 on dist-test-slave-kr9h"
server_key: "8af2871fb73ab3a9f8b162434e30be95"
server_key_iv: "97a8ac7f3e68ecf0c07193732c76588c"
server_key_version: "encryptionkey@0"
I20260619 12:28:34.577665 26467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "f8b304332541468aa39ec45ead315e8a"
format_stamp: "Formatted at 2026-06-19 12:28:34 on dist-test-slave-kr9h"
server_key: "8af2871fb73ab3a9f8b162434e30be95"
server_key_iv: "97a8ac7f3e68ecf0c07193732c76588c"
server_key_version: "encryptionkey@0"
I20260619 12:28:34.581815 26467 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 12:28:34.584628 26482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:34.586215 26467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:34.586365 26467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "f8b304332541468aa39ec45ead315e8a"
format_stamp: "Formatted at 2026-06-19 12:28:34 on dist-test-slave-kr9h"
server_key: "8af2871fb73ab3a9f8b162434e30be95"
server_key_iv: "97a8ac7f3e68ecf0c07193732c76588c"
server_key_version: "encryptionkey@0"
I20260619 12:28:34.586491 26467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:34.604048 26467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:34.607852 26467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:34.608088 26467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:34.608747 26467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:34.609783 26467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:34.609854 26467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:34.609946 26467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:34.609987 26467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:34.619676 26467 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:34005
I20260619 12:28:34.619700 26595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:34005 every 8 connection(s)
I20260619 12:28:34.620659 26467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:28:34.624939 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 26467
I20260619 12:28:34.625099 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:28:34.625473 7071 external_mini_cluster.cc:1428] Setting key a0d8ad359d109983d29b4869641a94bf
Jun 19 12:28:34 dist-test-slave-kr9h krb5kdc[26226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872114, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.194@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.195@KRBTEST.COM; defaulting to no policy
I20260619 12:28:34.637526 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.622814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:34565 (local address 127.6.231.254:35033)
0619 12:28:34.622963 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:34.622966 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:34.624192 (+ 1226us) server_negotiation.cc:407] Connection header received
0619 12:28:34.625516 (+ 1324us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:34.625534 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:34.625610 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:34.625685 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:34.627328 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.627850 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.629160 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.629395 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.629539 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.632851 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:34.632858 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:34.632863 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:34.632900 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:34.635028 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:34.635624 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:34.635631 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:34.635634 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:34.635745 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:34.636138 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:34.636141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:34.636143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:34.636415 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:34.636586 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:34.637190 (+ 604us) server_negotiation.cc:299] Negotiation successful
0619 12:28:34.637322 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 12:28:34.638696 26598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.623183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35033 (local address 127.6.231.194:34565)
0619 12:28:34.623990 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:34.624040 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:34.625161 (+ 1121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:34.625908 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:34.625918 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:34.626443 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:34.627129 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.627151 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.628021 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.628024 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.629002 (+ 978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.629012 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.629602 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.629606 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.630042 (+ 436us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.630812 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:28:34.630831 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:34.632589 (+ 1758us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:34.635178 (+ 2589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:34.635184 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:34.635201 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:34.635509 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:34.635866 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:34.635869 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:34.635870 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:34.636034 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:34.636622 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:34.636633 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:34.636915 (+ 282us) client_negotiation.cc:769] Sending connection context
0619 12:28:34.637166 (+ 251us) client_negotiation.cc:240] Negotiation successful
0619 12:28:34.637717 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":600,"thread_start_us":137,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.195@KRBTEST.COM" created.
I20260619 12:28:34.640283 26596 heartbeater.cc:347] Connected to a master server at 127.6.231.254:35033
I20260619 12:28:34.640579 26596 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:34.641309 26596 heartbeater.cc:511] Master 127.6.231.254:35033 requested a full tablet report, sending...
I20260619 12:28:34.642735 26271 ts_manager.cc:194] Registered new tserver with Master: f8b304332541468aa39ec45ead315e8a (127.6.231.194:34005)
I20260619 12:28:34.643487 26271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.194@KRBTEST.COM'} at 127.6.231.194:34565
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
Entry for principal kudu/127.6.231.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab.
I20260619 12:28:34.660539 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35033
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:43375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:34.789772 26601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:34.790126 26601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:34.790179 26601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:34.794673 26601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:34.794792 26601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:34.794866 26601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:28:34.800671 26601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:43375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.195.keytab
--principal=kudu/127.6.231.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:35033
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:34.801905 26601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:34.802983 26601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:34.813910 26609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:34.813910 26607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:34.814121 26606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:34.814430 26601 server_base.cc:1034] running on GCE node
I20260619 12:28:34.814893 26601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:34.815636 26601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:34.816874 26601 hybrid_clock.cc:648] HybridClock initialized: now 1781872114816841 us; error 45 us; skew 500 ppm
Jun 19 12:28:34 dist-test-slave-kr9h krb5kdc[26226](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872114, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:34.820142 26601 init.cc:377] Logged in from keytab as kudu/127.6.231.195@KRBTEST.COM (short username kudu)
I20260619 12:28:34.821241 26601 webserver.cc:460] Webserver started at http://127.6.231.195:45683/ using document root <none> and password file <none>
I20260619 12:28:34.821954 26601 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:34.822052 26601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:34.822288 26601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:34.824518 26601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "f4e7ea01e1ff4a049e9830785367d145"
format_stamp: "Formatted at 2026-06-19 12:28:34 on dist-test-slave-kr9h"
server_key: "4a780ebf40df01090e0267520037ee08"
server_key_iv: "9736a29838deee7b3c1293059541709e"
server_key_version: "encryptionkey@0"
I20260619 12:28:34.825169 26601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "f4e7ea01e1ff4a049e9830785367d145"
format_stamp: "Formatted at 2026-06-19 12:28:34 on dist-test-slave-kr9h"
server_key: "4a780ebf40df01090e0267520037ee08"
server_key_iv: "9736a29838deee7b3c1293059541709e"
server_key_version: "encryptionkey@0"
I20260619 12:28:34.829752 26601 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:28:34.833150 26616 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:34.835177 26601 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260619 12:28:34.835407 26601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "f4e7ea01e1ff4a049e9830785367d145"
format_stamp: "Formatted at 2026-06-19 12:28:34 on dist-test-slave-kr9h"
server_key: "4a780ebf40df01090e0267520037ee08"
server_key_iv: "9736a29838deee7b3c1293059541709e"
server_key_version: "encryptionkey@0"
I20260619 12:28:34.835582 26601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:34.856258 26601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:34.860275 26601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:34.860476 26601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:34.861068 26601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:34.862637 26601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:34.862703 26601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:34.862753 26601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:34.862774 26601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:34.873469 26601 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:42927
I20260619 12:28:34.873487 26729 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:42927 every 8 connection(s)
I20260619 12:28:34.874559 26601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:28:34.878527 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 26601
I20260619 12:28:34.878674 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:28:34.879052 7071 external_mini_cluster.cc:1428] Setting key 605224956af52b2324284d782a1dc422
Jun 19 12:28:34 dist-test-slave-kr9h krb5kdc[26226](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872114, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.195@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:28:34.890633 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.876817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:43867 (local address 127.6.231.254:35033)
0619 12:28:34.877007 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:34.877012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:34.877945 (+ 933us) server_negotiation.cc:407] Connection header received
0619 12:28:34.879033 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:34.879037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:34.879111 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:34.879219 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:34.880846 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.881653 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.882890 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.883135 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.883228 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.885973 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:34.885976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:34.885978 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:34.886005 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:34.887906 (+ 1901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:34.888616 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:34.888624 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:34.888627 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:34.888752 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:34.889267 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:34.889270 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:34.889271 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:34.889532 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:34.889620 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:34.890395 (+ 775us) server_negotiation.cc:299] Negotiation successful
0619 12:28:34.890504 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260619 12:28:34.891733 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.877196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:35033 (local address 127.6.231.195:43867)
0619 12:28:34.877781 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:34.877828 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:34.878816 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:34.879458 (+ 642us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:34.879471 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:34.880014 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:34.880689 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.880707 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.881782 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.881785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.882728 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.882761 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.883294 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.883297 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.883458 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.884066 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:28:34.884084 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:34.885782 (+ 1698us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:34.888111 (+ 2329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:34.888119 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:34.888144 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:34.888441 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:34.888924 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:28:34.888927 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:28:34.888929 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:28:34.889103 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:28:34.889641 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:34.889648 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:34.890082 (+ 434us) client_negotiation.cc:769] Sending connection context
0619 12:28:34.890339 (+ 257us) client_negotiation.cc:240] Negotiation successful
0619 12:28:34.890644 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":176,"threads_started":1}
I20260619 12:28:34.892984 26730 heartbeater.cc:347] Connected to a master server at 127.6.231.254:35033
I20260619 12:28:34.893231 26730 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:34.893857 26730 heartbeater.cc:511] Master 127.6.231.254:35033 requested a full tablet report, sending...
I20260619 12:28:34.894922 26270 ts_manager.cc:194] Registered new tserver with Master: f4e7ea01e1ff4a049e9830785367d145 (127.6.231.195:42927)
I20260619 12:28:34.895532 26270 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.6.231.195@KRBTEST.COM'} at 127.6.231.195:43867
I20260619 12:28:34.905864 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:28:34.915536 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.908015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60870 (local address 127.6.231.254:35033)
0619 12:28:34.908155 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:34.908159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:34.908335 (+ 176us) server_negotiation.cc:407] Connection header received
0619 12:28:34.908441 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:34.908443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:34.908486 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:34.908560 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:34.909267 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.909885 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.910800 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.911040 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.911125 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.912514 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:34.912517 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:34.912519 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:34.912544 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:34.913895 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:34.914321 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:34.914324 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:34.914325 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:34.914410 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:34.914670 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:34.914672 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:34.914673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:34.914910 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:34.915020 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:34.915307 (+ 287us) server_negotiation.cc:299] Negotiation successful
0619 12:28:34.915411 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:28:34.919171 26270 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:28:34.922657 26270 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:28:34.939469 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.934021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52668 (local address 127.6.231.193:35751)
0619 12:28:34.934423 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:34.934431 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:34.934453 (+ 22us) server_negotiation.cc:407] Connection header received
0619 12:28:34.934526 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:34.934532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:34.934725 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:34.934887 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:34.935291 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.936397 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.938089 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.938656 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.938790 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.938937 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:34.939168 (+ 231us) server_negotiation.cc:299] Negotiation successful
0619 12:28:34.939342 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":286,"thread_start_us":105,"threads_started":1}
I20260619 12:28:34.939461 26741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.933130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:35751 (local address 127.0.0.1:52668)
0619 12:28:34.933604 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:34.933665 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:34.933806 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:34.935027 (+ 1221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:34.935031 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:34.935052 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:34.935143 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.935158 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.936483 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.936489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.937939 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.937949 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.938799 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.938802 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.938918 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.938990 (+ 72us) client_negotiation.cc:769] Sending connection context
0619 12:28:34.939133 (+ 143us) client_negotiation.cc:240] Negotiation successful
0619 12:28:34.939277 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":50,"thread_start_us":116,"threads_started":1}
I20260619 12:28:34.939790 26742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.932853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48070 (local address 127.6.231.194:34005)
0619 12:28:34.933562 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:34.933576 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:34.933955 (+ 379us) server_negotiation.cc:407] Connection header received
0619 12:28:34.934050 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:34.934056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:34.934318 (+ 262us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:34.934483 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:34.934828 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.935983 (+ 1155us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.938073 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.938895 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.938999 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.939122 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:34.939536 (+ 414us) server_negotiation.cc:299] Negotiation successful
0619 12:28:34.939644 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":125,"threads_started":1}
I20260619 12:28:34.939788 26744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.933566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:42927 (local address 127.0.0.1:40316)
0619 12:28:34.934079 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:34.934091 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:34.934169 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:34.934833 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:34.934835 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:34.934852 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:34.934942 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.934948 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.936306 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.936309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.937663 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.937673 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.939021 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.939025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.939115 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.939139 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 12:28:34.939206 (+ 67us) client_negotiation.cc:240] Negotiation successful
0619 12:28:34.939288 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"mutex_wait_us":183,"thread_start_us":103,"threads_started":1}
I20260619 12:28:34.940153 26743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.932709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:34005 (local address 127.0.0.1:48070)
0619 12:28:34.933604 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:34.933666 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:34.933806 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:34.934493 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:34.934500 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:34.934524 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:34.934678 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.934686 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.936225 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.936231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.937938 (+ 1707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.937949 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.939196 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.939202 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.939283 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.939302 (+ 19us) client_negotiation.cc:769] Sending connection context
0619 12:28:34.939370 (+ 68us) client_negotiation.cc:240] Negotiation successful
0619 12:28:34.939427 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"mutex_wait_us":196,"thread_start_us":88,"threads_started":1}
I20260619 12:28:34.940349 26745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.933976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40316 (local address 127.6.231.195:42927)
0619 12:28:34.934393 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:34.934427 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:34.934453 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:28:34.934521 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:34.934526 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:34.934705 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:34.934873 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:34.935022 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.936134 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.937793 (+ 1659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.938649 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.938767 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.939329 (+ 562us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:34.940021 (+ 692us) server_negotiation.cc:299] Negotiation successful
0619 12:28:34.940193 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":97,"threads_started":1}
I20260619 12:28:34.943379 26396 tablet_service.cc:1467] Processing CreateTablet for tablet 58083168b5c64238bea5ede0e6956bcb (DEFAULT_TABLE table=test-table [id=4e0266b592314a64a6513b62dea316fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:34.943375 26664 tablet_service.cc:1467] Processing CreateTablet for tablet 58083168b5c64238bea5ede0e6956bcb (DEFAULT_TABLE table=test-table [id=4e0266b592314a64a6513b62dea316fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:34.943378 26530 tablet_service.cc:1467] Processing CreateTablet for tablet 58083168b5c64238bea5ede0e6956bcb (DEFAULT_TABLE table=test-table [id=4e0266b592314a64a6513b62dea316fc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:34.944542 26664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58083168b5c64238bea5ede0e6956bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:34.944536 26530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58083168b5c64238bea5ede0e6956bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:34.944536 26396 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58083168b5c64238bea5ede0e6956bcb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:34.951015 26747 tablet_bootstrap.cc:492] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145: Bootstrap starting.
I20260619 12:28:34.952607 26749 tablet_bootstrap.cc:492] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a: Bootstrap starting.
I20260619 12:28:34.953289 26747 tablet_bootstrap.cc:654] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:34.954440 26747 log.cc:824] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:34.954936 26749 tablet_bootstrap.cc:654] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:34.955704 26749 log.cc:824] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:34.956326 26748 tablet_bootstrap.cc:492] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a: Bootstrap starting.
I20260619 12:28:34.956547 26747 tablet_bootstrap.cc:492] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145: No bootstrap required, opened a new log
I20260619 12:28:34.956738 26747 ts_tablet_manager.cc:1397] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 12:28:34.957674 26749 tablet_bootstrap.cc:492] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a: No bootstrap required, opened a new log
I20260619 12:28:34.957873 26749 ts_tablet_manager.cc:1397] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260619 12:28:34.958370 26748 tablet_bootstrap.cc:654] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:34.959026 26748 log.cc:824] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:34.960901 26748 tablet_bootstrap.cc:492] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a: No bootstrap required, opened a new log
I20260619 12:28:34.960835 26747 raft_consensus.cc:348] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b304332541468aa39ec45ead315e8a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34005 } } peers { permanent_uuid: "1b34602eff2a47358ea3d16afccb5f4a" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35751 } } peers { permanent_uuid: "f4e7ea01e1ff4a049e9830785367d145" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42927 } }
I20260619 12:28:34.961088 26748 ts_tablet_manager.cc:1397] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260619 12:28:34.961131 26747 raft_consensus.cc:374] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:34.961192 26747 raft_consensus.cc:729] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4e7ea01e1ff4a049e9830785367d145, State: Initialized, Role: FOLLOWER
I20260619 12:28:34.961741 26747 consensus_queue.cc:260] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b304332541468aa39ec45ead315e8a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34005 } } peers { permanent_uuid: "1b34602eff2a47358ea3d16afccb5f4a" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35751 } } peers { permanent_uuid: "f4e7ea01e1ff4a049e9830785367d145" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42927 } }
I20260619 12:28:34.961702 26749 raft_consensus.cc:348] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b304332541468aa39ec45ead315e8a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34005 } } peers { permanent_uuid: "1b34602eff2a47358ea3d16afccb5f4a" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35751 } } peers { permanent_uuid: "f4e7ea01e1ff4a049e9830785367d145" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42927 } }
I20260619 12:28:34.961977 26749 raft_consensus.cc:374] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:34.962047 26749 raft_consensus.cc:729] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8b304332541468aa39ec45ead315e8a, State: Initialized, Role: FOLLOWER
I20260619 12:28:34.962462 26749 consensus_queue.cc:260] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b304332541468aa39ec45ead315e8a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34005 } } peers { permanent_uuid: "1b34602eff2a47358ea3d16afccb5f4a" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35751 } } peers { permanent_uuid: "f4e7ea01e1ff4a049e9830785367d145" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42927 } }
I20260619 12:28:34.962826 26747 ts_tablet_manager.cc:1428] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 12:28:34.963183 26730 heartbeater.cc:503] Master 127.6.231.254:35033 was elected leader, sending a full tablet report...
I20260619 12:28:34.963225 26596 heartbeater.cc:503] Master 127.6.231.254:35033 was elected leader, sending a full tablet report...
I20260619 12:28:34.963642 26749 ts_tablet_manager.cc:1428] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 12:28:34.965481 26748 raft_consensus.cc:348] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b304332541468aa39ec45ead315e8a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34005 } } peers { permanent_uuid: "1b34602eff2a47358ea3d16afccb5f4a" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35751 } } peers { permanent_uuid: "f4e7ea01e1ff4a049e9830785367d145" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42927 } }
I20260619 12:28:34.965777 26748 raft_consensus.cc:374] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:34.965845 26748 raft_consensus.cc:729] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b34602eff2a47358ea3d16afccb5f4a, State: Initialized, Role: FOLLOWER
I20260619 12:28:34.966313 26748 consensus_queue.cc:260] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b304332541468aa39ec45ead315e8a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34005 } } peers { permanent_uuid: "1b34602eff2a47358ea3d16afccb5f4a" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35751 } } peers { permanent_uuid: "f4e7ea01e1ff4a049e9830785367d145" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42927 } }
I20260619 12:28:34.966881 26753 raft_consensus.cc:482] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:28:34.967072 26753 raft_consensus.cc:504] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b304332541468aa39ec45ead315e8a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34005 } } peers { permanent_uuid: "1b34602eff2a47358ea3d16afccb5f4a" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35751 } } peers { permanent_uuid: "f4e7ea01e1ff4a049e9830785367d145" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42927 } }
I20260619 12:28:34.968010 26462 heartbeater.cc:503] Master 127.6.231.254:35033 was elected leader, sending a full tablet report...
I20260619 12:28:34.969470 26753 leader_election.cc:288] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f8b304332541468aa39ec45ead315e8a (127.6.231.194:34005), 1b34602eff2a47358ea3d16afccb5f4a (127.6.231.193:35751)
I20260619 12:28:34.969913 26748 ts_tablet_manager.cc:1428] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260619 12:28:34.977360 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.969529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:58245 (local address 127.6.231.193:35751)
0619 12:28:34.969643 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:34.969648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:34.973047 (+ 3399us) server_negotiation.cc:407] Connection header received
0619 12:28:34.973217 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:34.973220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:34.973273 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:34.973315 (+ 42us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:34.973717 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.974467 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.976032 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.976648 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.976738 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.976791 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:34.977126 (+ 335us) server_negotiation.cc:299] Negotiation successful
0619 12:28:34.977210 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260619 12:28:34.979789 26756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.969200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:35751 (local address 127.6.231.195:58245)
0619 12:28:34.972960 (+ 3760us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:34.972976 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:34.973115 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:34.973464 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:34.973467 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:34.973482 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:34.973606 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.973611 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.974610 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.974613 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.975893 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.975904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.976886 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.976889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.976990 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.977017 (+ 27us) client_negotiation.cc:769] Sending connection context
0619 12:28:34.977617 (+ 600us) client_negotiation.cc:240] Negotiation successful
0619 12:28:34.979589 (+ 1972us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3675,"thread_start_us":128,"threads_started":1}
I20260619 12:28:34.980677 26416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58083168b5c64238bea5ede0e6956bcb" candidate_uuid: "f4e7ea01e1ff4a049e9830785367d145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b34602eff2a47358ea3d16afccb5f4a" is_pre_election: true
I20260619 12:28:34.981045 26416 raft_consensus.cc:2454] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4e7ea01e1ff4a049e9830785367d145 in term 0.
I20260619 12:28:34.981618 26619 leader_election.cc:302] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1b34602eff2a47358ea3d16afccb5f4a, f4e7ea01e1ff4a049e9830785367d145; no voters:
I20260619 12:28:34.981948 26753 raft_consensus.cc:2781] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:28:34.982064 26753 raft_consensus.cc:482] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:28:34.982112 26753 raft_consensus.cc:3037] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:34.982467 26732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.968556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:34005 (local address 127.6.231.195:36491)
0619 12:28:34.968719 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:34.968734 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:34.968828 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:34.977729 (+ 8901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:34.977733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:34.977771 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:34.977869 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.977876 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.979491 (+ 1615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.979494 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.980758 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:34.980785 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.981976 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.981980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:34.982093 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.982124 (+ 31us) client_negotiation.cc:769] Sending connection context
0619 12:28:34.982207 (+ 83us) client_negotiation.cc:240] Negotiation successful
0619 12:28:34.982296 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":90,"mutex_wait_us":25}
I20260619 12:28:34.982853 26742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:34.969226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:36491 (local address 127.6.231.194:34005)
0619 12:28:34.969689 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:34.969695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:34.969715 (+ 20us) server_negotiation.cc:407] Connection header received
0619 12:28:34.969767 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:34.969770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:34.969818 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:34.969900 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:34.978539 (+ 8639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.979302 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.980952 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:34.981677 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:34.981838 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:34.981909 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:34.982633 (+ 724us) server_negotiation.cc:299] Negotiation successful
0619 12:28:34.982722 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260619 12:28:34.983479 26753 raft_consensus.cc:504] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b304332541468aa39ec45ead315e8a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34005 } } peers { permanent_uuid: "1b34602eff2a47358ea3d16afccb5f4a" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35751 } } peers { permanent_uuid: "f4e7ea01e1ff4a049e9830785367d145" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42927 } }
I20260619 12:28:34.983496 26550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58083168b5c64238bea5ede0e6956bcb" candidate_uuid: "f4e7ea01e1ff4a049e9830785367d145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b304332541468aa39ec45ead315e8a" is_pre_election: true
I20260619 12:28:34.983831 26550 raft_consensus.cc:2454] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f4e7ea01e1ff4a049e9830785367d145 in term 0.
I20260619 12:28:34.983908 26753 leader_election.cc:288] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [CANDIDATE]: Term 1 election: Requested vote from peers f8b304332541468aa39ec45ead315e8a (127.6.231.194:34005), 1b34602eff2a47358ea3d16afccb5f4a (127.6.231.193:35751)
I20260619 12:28:34.984253 26550 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58083168b5c64238bea5ede0e6956bcb" candidate_uuid: "f4e7ea01e1ff4a049e9830785367d145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8b304332541468aa39ec45ead315e8a"
I20260619 12:28:34.984304 26416 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58083168b5c64238bea5ede0e6956bcb" candidate_uuid: "f4e7ea01e1ff4a049e9830785367d145" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b34602eff2a47358ea3d16afccb5f4a"
I20260619 12:28:34.984385 26550 raft_consensus.cc:3037] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:34.984476 26416 raft_consensus.cc:3037] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:34.985737 26550 raft_consensus.cc:2454] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e7ea01e1ff4a049e9830785367d145 in term 1.
I20260619 12:28:34.985730 26416 raft_consensus.cc:2454] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f4e7ea01e1ff4a049e9830785367d145 in term 1.
I20260619 12:28:34.986064 26620 leader_election.cc:302] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f4e7ea01e1ff4a049e9830785367d145, f8b304332541468aa39ec45ead315e8a; no voters:
I20260619 12:28:34.986248 26753 raft_consensus.cc:2781] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:34.986492 26753 raft_consensus.cc:686] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [term 1 LEADER]: Becoming Leader. State: Replica: f4e7ea01e1ff4a049e9830785367d145, State: Running, Role: LEADER
I20260619 12:28:34.986850 26753 consensus_queue.cc:237] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b304332541468aa39ec45ead315e8a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34005 } } peers { permanent_uuid: "1b34602eff2a47358ea3d16afccb5f4a" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35751 } } peers { permanent_uuid: "f4e7ea01e1ff4a049e9830785367d145" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42927 } }
I20260619 12:28:34.990734 26270 catalog_manager.cc:5495] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 reported cstate change: term changed from 0 to 1, leader changed from <none> to f4e7ea01e1ff4a049e9830785367d145 (127.6.231.195). New cstate: current_term: 1 leader_uuid: "f4e7ea01e1ff4a049e9830785367d145" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f8b304332541468aa39ec45ead315e8a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34005 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b34602eff2a47358ea3d16afccb5f4a" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f4e7ea01e1ff4a049e9830785367d145" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 42927 } health_report { overall_health: HEALTHY } } }
I20260619 12:28:35.005348 26745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.001670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40320 (local address 127.6.231.195:42927)
0619 12:28:35.001840 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:35.001845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:35.001865 (+ 20us) server_negotiation.cc:407] Connection header received
0619 12:28:35.001937 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:35.001939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:35.001993 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:35.002106 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:28:35.002391 (+ 285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.002907 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.003835 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.004065 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.004175 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.004495 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:28:35.004939 (+ 444us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:28:35.005087 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:35.005128 (+ 41us) server_negotiation.cc:299] Negotiation successful
0619 12:28:35.005188 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260619 12:28:35.011972 26416 raft_consensus.cc:1261] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a [term 1 FOLLOWER]: Refusing update from remote peer f4e7ea01e1ff4a049e9830785367d145: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:28:35.011972 26550 raft_consensus.cc:1261] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a [term 1 FOLLOWER]: Refusing update from remote peer f4e7ea01e1ff4a049e9830785367d145: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:28:35.012807 26757 consensus_queue.cc:1035] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b34602eff2a47358ea3d16afccb5f4a" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 35751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:28:35.013026 26753 consensus_queue.cc:1035] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8b304332541468aa39ec45ead315e8a" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 34005 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:28:35.022022 26762 mvcc.cc:205] Tried to move back new op lower bound from 7298548183080693760 to 7298548182988906496. Current Snapshot: MvccSnapshot[applied={T|T < 7298548183080693760}]
I20260619 12:28:35.029058 26761 mvcc.cc:205] Tried to move back new op lower bound from 7298548183080693760 to 7298548182988906496. Current Snapshot: MvccSnapshot[applied={T|T < 7298548183080693760 or (T in {7298548183080693760})}]
I20260619 12:28:35.030130 26270 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:28:35.030319 26270 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:28:35.032992 26270 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac449efd133b457dafdf687d807d7d4f: Sending DeleteTablet for 3 replicas of tablet 58083168b5c64238bea5ede0e6956bcb
I20260619 12:28:35.033905 26396 tablet_service.cc:1514] Processing DeleteTablet for tablet 58083168b5c64238bea5ede0e6956bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:35 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:52668
I20260619 12:28:35.033905 26530 tablet_service.cc:1514] Processing DeleteTablet for tablet 58083168b5c64238bea5ede0e6956bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:35 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:48070
I20260619 12:28:35.034487 26769 tablet_replica.cc:330] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a: stopping tablet replica
I20260619 12:28:35.034477 26664 tablet_service.cc:1514] Processing DeleteTablet for tablet 58083168b5c64238bea5ede0e6956bcb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:35 UTC) from {username='kudu', principal='kudu/127.6.231.254@KRBTEST.COM'} at 127.0.0.1:40316
I20260619 12:28:35.034796 26770 tablet_replica.cc:330] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a: stopping tablet replica
I20260619 12:28:35.034952 26771 tablet_replica.cc:330] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145: stopping tablet replica
I20260619 12:28:35.035247 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 26333
I20260619 12:28:35.035245 26771 raft_consensus.cc:2229] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [term 1 LEADER]: Raft consensus shutting down.
I20260619 12:28:35.035511 26770 raft_consensus.cc:2229] T 58083168b5c64238bea5ede0e6956bcb P 1b34602eff2a47358ea3d16afccb5f4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:28:35.035259 26769 raft_consensus.cc:2229] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:28:35.035996 26769 raft_consensus.cc:2258] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:28:35.037533 26771 raft_consensus.cc:2258] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:28:35.037851 26769 ts_tablet_manager.cc:1905] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:28:35.041523 26769 ts_tablet_manager.cc:1918] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:28:35.041640 26769 log.cc:1196] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/58083168b5c64238bea5ede0e6956bcb
I20260619 12:28:35.041993 26769 ts_tablet_manager.cc:1939] T 58083168b5c64238bea5ede0e6956bcb P f8b304332541468aa39ec45ead315e8a: Deleting consensus metadata
I20260619 12:28:35.043470 26258 catalog_manager.cc:4841] TS f8b304332541468aa39ec45ead315e8a (127.6.231.194:34005): tablet 58083168b5c64238bea5ede0e6956bcb (table test-table [id=4e0266b592314a64a6513b62dea316fc]) successfully deleted
W20260619 12:28:35.043821 26257 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:35751 (error 108)
I20260619 12:28:35.044040 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 26467
I20260619 12:28:35.045727 26771 ts_tablet_manager.cc:1905] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:28:35.048023 26771 ts_tablet_manager.cc:1918] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:28:35.048125 26771 log.cc:1196] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/58083168b5c64238bea5ede0e6956bcb
I20260619 12:28:35.048475 26771 ts_tablet_manager.cc:1939] T 58083168b5c64238bea5ede0e6956bcb P f4e7ea01e1ff4a049e9830785367d145: Deleting consensus metadata
I20260619 12:28:35.048861 26743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.048476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:35751 (local address 127.0.0.1:52672)
0619 12:28:35.048689 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:35.048772 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:35751: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260619 12:28:35.049165 26257 catalog_manager.cc:4584] TS 1b34602eff2a47358ea3d16afccb5f4a (127.6.231.193:35751): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 58083168b5c64238bea5ede0e6956bcb: Network error: Client connection negotiation failed: client connection to 127.6.231.193:35751: connect: Connection refused (error 111)
I20260619 12:28:35.049851 26257 catalog_manager.cc:4841] TS f4e7ea01e1ff4a049e9830785367d145 (127.6.231.195:42927): tablet 58083168b5c64238bea5ede0e6956bcb (table test-table [id=4e0266b592314a64a6513b62dea316fc]) successfully deleted
I20260619 12:28:35.051442 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 26601
I20260619 12:28:35.056380 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 26239
2026-06-19T12:28:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2395 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-06-19T12:28:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:35Z Disabled control of system clock
I20260619 12:28:35.087005 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:44327
--webserver_interface=127.6.231.254
--webserver_port=0
--builtin_ntp_servers=127.6.231.212:43705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:44327
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:28:35.199658 26776 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:35.200009 26776 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:35.200083 26776 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:35.204233 26776 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:35.204352 26776 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:35.204382 26776 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:35.204425 26776 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:35.204461 26776 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:35.209542 26776 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:43705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:44327
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:44327
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26776
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:35.210836 26776 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:35.212306 26776 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:35.220963 26781 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:35.220959 26782 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:35.221012 26784 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:35.221588 26776 server_base.cc:1034] running on GCE node
I20260619 12:28:35.222150 26776 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:35.223377 26776 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:35.224548 26776 hybrid_clock.cc:648] HybridClock initialized: now 1781872115224525 us; error 39 us; skew 500 ppm
I20260619 12:28:35.230180 26776 webserver.cc:460] Webserver started at http://127.6.231.254:33181/ using document root <none> and password file <none>
I20260619 12:28:35.230865 26776 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:35.230955 26776 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:35.231249 26776 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:35.234123 26776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "2e6bd7797e274d94846ee1adc10fb92f"
format_stamp: "Formatted at 2026-06-19 12:28:35 on dist-test-slave-kr9h"
server_key: "b2a58ebcd408f315458c52f6549a16d6"
server_key_iv: "ab4885ed82de14ceb67b6dc0fac37198"
server_key_version: "encryptionkey@0"
I20260619 12:28:35.234719 26776 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "2e6bd7797e274d94846ee1adc10fb92f"
format_stamp: "Formatted at 2026-06-19 12:28:35 on dist-test-slave-kr9h"
server_key: "b2a58ebcd408f315458c52f6549a16d6"
server_key_iv: "ab4885ed82de14ceb67b6dc0fac37198"
server_key_version: "encryptionkey@0"
I20260619 12:28:35.239969 26776 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260619 12:28:35.243338 26790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:35.244768 26776 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:28:35.244938 26776 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "2e6bd7797e274d94846ee1adc10fb92f"
format_stamp: "Formatted at 2026-06-19 12:28:35 on dist-test-slave-kr9h"
server_key: "b2a58ebcd408f315458c52f6549a16d6"
server_key_iv: "ab4885ed82de14ceb67b6dc0fac37198"
server_key_version: "encryptionkey@0"
I20260619 12:28:35.245098 26776 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:35.259124 26776 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:35.259902 26776 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:35.260114 26776 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:35.267271 26776 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:44327
I20260619 12:28:35.267287 26842 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:44327 every 8 connection(s)
I20260619 12:28:35.268314 26776 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:35.271562 26843 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:35.273111 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 26776
I20260619 12:28:35.273249 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:35.273615 7071 external_mini_cluster.cc:1428] Setting key 988fa496fe22d93f6fa678dc7eb03cfc
I20260619 12:28:35.278836 26843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f: Bootstrap starting.
I20260619 12:28:35.281273 26843 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:35.282089 26843 log.cc:824] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:35.284029 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.275362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53160 (local address 127.6.231.254:44327)
0619 12:28:35.276143 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:35.276160 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:35.276200 (+ 40us) server_negotiation.cc:407] Connection header received
0619 12:28:35.276994 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:35.277019 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:35.277551 (+ 532us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:35.277926 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:35.278404 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.279586 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.280514 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.281011 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.281212 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.282190 (+ 978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:35.282196 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:35.282211 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:35.282666 (+ 455us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:35.282728 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:35.282750 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:35.282861 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:35.283008 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 12:28:35.283294 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":153,"threads_started":1}
I20260619 12:28:35.284241 26843 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f: No bootstrap required, opened a new log
I20260619 12:28:35.288205 26843 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e6bd7797e274d94846ee1adc10fb92f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 44327 } }
I20260619 12:28:35.288483 26843 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:35.288576 26843 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e6bd7797e274d94846ee1adc10fb92f, State: Initialized, Role: FOLLOWER
I20260619 12:28:35.289016 26843 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e6bd7797e274d94846ee1adc10fb92f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 44327 } }
I20260619 12:28:35.289168 26843 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:35.289254 26843 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:35.289376 26843 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:35.290601 26843 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e6bd7797e274d94846ee1adc10fb92f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 44327 } }
I20260619 12:28:35.290992 26843 leader_election.cc:302] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2e6bd7797e274d94846ee1adc10fb92f; no voters:
I20260619 12:28:35.291342 26843 leader_election.cc:288] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:35.291460 26848 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:35.291729 26848 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [term 1 LEADER]: Becoming Leader. State: Replica: 2e6bd7797e274d94846ee1adc10fb92f, State: Running, Role: LEADER
I20260619 12:28:35.292047 26848 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e6bd7797e274d94846ee1adc10fb92f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 44327 } }
I20260619 12:28:35.292624 26843 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:35.293761 26849 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2e6bd7797e274d94846ee1adc10fb92f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e6bd7797e274d94846ee1adc10fb92f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 44327 } } }
I20260619 12:28:35.293895 26849 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:35.294246 26850 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2e6bd7797e274d94846ee1adc10fb92f. Latest consensus state: current_term: 1 leader_uuid: "2e6bd7797e274d94846ee1adc10fb92f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2e6bd7797e274d94846ee1adc10fb92f" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 44327 } } }
I20260619 12:28:35.294394 26850 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:35.294474 26857 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:35.297796 26857 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:35.303754 26857 catalog_manager.cc:1347] Generated new cluster ID: 7fa5a6eee31a4fcb8d34b6cade2d4300
I20260619 12:28:35.303853 26857 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:35.312410 26857 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:35.313830 26857 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:35.326656 26857 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f: Generated new TSK 0
I20260619 12:28:35.327616 26857 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:35.333630 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.193:0
--local_ip_for_outbound_sockets=127.6.231.193
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:44327
--builtin_ntp_servers=127.6.231.212:43705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260619 12:28:35.456977 26867 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:35.457337 26867 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:35.457423 26867 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:35.461848 26867 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:35.461943 26867 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:35.462034 26867 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.193
I20260619 12:28:35.467283 26867 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:43705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.6.231.193
--webserver_port=0
--tserver_master_addrs=127.6.231.254:44327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26867
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.193
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:35.468492 26867 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:35.470018 26867 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:35.480029 26875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:35.480322 26867 server_base.cc:1034] running on GCE node
W20260619 12:28:35.480275 26873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:35.480275 26872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:35.480995 26867 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:35.481732 26867 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:35.482908 26867 hybrid_clock.cc:648] HybridClock initialized: now 1781872115482895 us; error 24 us; skew 500 ppm
I20260619 12:28:35.484952 26867 webserver.cc:460] Webserver started at http://127.6.231.193:37295/ using document root <none> and password file <none>
I20260619 12:28:35.485610 26867 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:35.485706 26867 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:35.485956 26867 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:35.488101 26867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/data/instance:
uuid: "4e986f71d9ea49b799d9fd1fa222ea8e"
format_stamp: "Formatted at 2026-06-19 12:28:35 on dist-test-slave-kr9h"
server_key: "bdb9890a93b65cf20c21d19b547a6a2f"
server_key_iv: "d442f5084ee4a4d712c222d90ded11fa"
server_key_version: "encryptionkey@0"
I20260619 12:28:35.488633 26867 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance:
uuid: "4e986f71d9ea49b799d9fd1fa222ea8e"
format_stamp: "Formatted at 2026-06-19 12:28:35 on dist-test-slave-kr9h"
server_key: "bdb9890a93b65cf20c21d19b547a6a2f"
server_key_iv: "d442f5084ee4a4d712c222d90ded11fa"
server_key_version: "encryptionkey@0"
I20260619 12:28:35.492606 26867 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260619 12:28:35.495551 26881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:35.496906 26867 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260619 12:28:35.497073 26867 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/wal
uuid: "4e986f71d9ea49b799d9fd1fa222ea8e"
format_stamp: "Formatted at 2026-06-19 12:28:35 on dist-test-slave-kr9h"
server_key: "bdb9890a93b65cf20c21d19b547a6a2f"
server_key_iv: "d442f5084ee4a4d712c222d90ded11fa"
server_key_version: "encryptionkey@0"
I20260619 12:28:35.497225 26867 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:35.521914 26867 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:35.522639 26867 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:35.522799 26867 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:35.523373 26867 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:35.524439 26867 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:35.524506 26867 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:35.524580 26867 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:35.524613 26867 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:35.533855 26867 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.193:39493
I20260619 12:28:35.533934 26994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.193:39493 every 8 connection(s)
I20260619 12:28:35.534875 26867 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/data/info.pb
I20260619 12:28:35.540894 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 26867
I20260619 12:28:35.541066 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-0/wal/instance
I20260619 12:28:35.541504 7071 external_mini_cluster.cc:1428] Setting key 9793a320b99c76d8260bfbb17e504005
I20260619 12:28:35.543653 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.194:0
--local_ip_for_outbound_sockets=127.6.231.194
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:44327
--builtin_ntp_servers=127.6.231.212:43705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:28:35.545601 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.536951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:35811 (local address 127.6.231.254:44327)
0619 12:28:35.537138 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:35.537143 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:35.538459 (+ 1316us) server_negotiation.cc:407] Connection header received
0619 12:28:35.539867 (+ 1408us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:35.539870 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:35.539928 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:35.540029 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:35.541038 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.541767 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.543219 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.543458 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.543582 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.544949 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:35.544956 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:35.544960 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:35.544980 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:35.545008 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:35.545022 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:35.545116 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:35.545364 (+ 248us) server_negotiation.cc:299] Negotiation successful
0619 12:28:35.545456 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260619 12:28:35.546905 26997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.537526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:44327 (local address 127.6.231.193:35811)
0619 12:28:35.538284 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:35.538328 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:35.539610 (+ 1282us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:35.540213 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:35.540222 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:35.540324 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:35.540860 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:35.540874 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.541907 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.541910 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:35.543083 (+ 1173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:35.543092 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.543627 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.543631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:35.543863 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.544640 (+ 777us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:28:35.544642 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:35.544689 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:28:35.544768 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:35.545163 (+ 395us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:35.545174 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:35.545182 (+ 8us) client_negotiation.cc:769] Sending connection context
0619 12:28:35.545338 (+ 156us) client_negotiation.cc:240] Negotiation successful
0619 12:28:35.545737 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":135,"threads_started":1}
I20260619 12:28:35.548321 26995 heartbeater.cc:347] Connected to a master server at 127.6.231.254:44327
I20260619 12:28:35.548629 26995 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:35.549232 26995 heartbeater.cc:511] Master 127.6.231.254:44327 requested a full tablet report, sending...
I20260619 12:28:35.550925 26807 ts_manager.cc:194] Registered new tserver with Master: 4e986f71d9ea49b799d9fd1fa222ea8e (127.6.231.193:39493)
I20260619 12:28:35.552085 26807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.231.193:35811
W20260619 12:28:35.659752 26998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:35.660048 26998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:35.660087 26998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:35.664431 26998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:35.664508 26998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:35.664566 26998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.194
I20260619 12:28:35.669690 26998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:43705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.6.231.194
--webserver_port=0
--tserver_master_addrs=127.6.231.254:44327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.194
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:35.670872 26998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:35.672214 26998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:28:35.682092 26998 server_base.cc:1034] running on GCE node
W20260619 12:28:35.682006 27003 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:35.682040 27006 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:35.682024 27004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:35.682740 26998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:35.683429 26998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:35.684623 26998 hybrid_clock.cc:648] HybridClock initialized: now 1781872115684607 us; error 36 us; skew 500 ppm
I20260619 12:28:35.686667 26998 webserver.cc:460] Webserver started at http://127.6.231.194:44889/ using document root <none> and password file <none>
I20260619 12:28:35.687284 26998 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:35.687382 26998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:35.687618 26998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:35.689873 26998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/data/instance:
uuid: "8dc809a2ec53461892463a0f4b13e6d6"
format_stamp: "Formatted at 2026-06-19 12:28:35 on dist-test-slave-kr9h"
server_key: "8d8a182c4f1503cba338696d913a4b55"
server_key_iv: "78b9b187f49137af571a19e1f0499627"
server_key_version: "encryptionkey@0"
I20260619 12:28:35.690496 26998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance:
uuid: "8dc809a2ec53461892463a0f4b13e6d6"
format_stamp: "Formatted at 2026-06-19 12:28:35 on dist-test-slave-kr9h"
server_key: "8d8a182c4f1503cba338696d913a4b55"
server_key_iv: "78b9b187f49137af571a19e1f0499627"
server_key_version: "encryptionkey@0"
I20260619 12:28:35.694530 26998 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:28:35.697182 27012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:35.698647 26998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:28:35.698788 26998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/wal
uuid: "8dc809a2ec53461892463a0f4b13e6d6"
format_stamp: "Formatted at 2026-06-19 12:28:35 on dist-test-slave-kr9h"
server_key: "8d8a182c4f1503cba338696d913a4b55"
server_key_iv: "78b9b187f49137af571a19e1f0499627"
server_key_version: "encryptionkey@0"
I20260619 12:28:35.698917 26998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:35.717682 26998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:35.718451 26998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:35.718689 26998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:35.719328 26998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:35.720321 26998 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:35.720368 26998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:35.720434 26998 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:35.720466 26998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:35.729176 26998 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.194:35427
I20260619 12:28:35.729262 27125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.194:35427 every 8 connection(s)
I20260619 12:28:35.730226 26998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/data/info.pb
I20260619 12:28:35.731154 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 26998
I20260619 12:28:35.731294 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/wal/instance
I20260619 12:28:35.731626 7071 external_mini_cluster.cc:1428] Setting key a7a03206653f29e189124347bb10617f
I20260619 12:28:35.733896 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.6.231.195:0
--local_ip_for_outbound_sockets=127.6.231.195
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:44327
--builtin_ntp_servers=127.6.231.212:43705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260619 12:28:35.740918 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.732384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.194:40113 (local address 127.6.231.254:44327)
0619 12:28:35.732544 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:35.732548 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:35.734029 (+ 1481us) server_negotiation.cc:407] Connection header received
0619 12:28:35.735996 (+ 1967us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:35.735999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:35.736055 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:35.736138 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:35.737081 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.737716 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.738890 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.739111 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.739225 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.740389 (+ 1164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:35.740392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:35.740395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:35.740416 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:35.740433 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:35.740441 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:35.740517 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:35.740694 (+ 177us) server_negotiation.cc:299] Negotiation successful
0619 12:28:35.740761 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260619 12:28:35.741873 27128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.732971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:44327 (local address 127.6.231.194:40113)
0619 12:28:35.733858 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:35.733915 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:35.735772 (+ 1857us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:35.736322 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:35.736329 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:35.736422 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:35.736937 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:35.736950 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.737853 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.737857 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:35.738762 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:35.738771 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.739271 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.739275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:35.739449 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.740106 (+ 657us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:28:35.740107 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:35.740151 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:28:35.740218 (+ 67us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:35.740531 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:35.740541 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:35.740553 (+ 12us) client_negotiation.cc:769] Sending connection context
0619 12:28:35.740682 (+ 129us) client_negotiation.cc:240] Negotiation successful
0619 12:28:35.740983 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":149,"threads_started":1}
I20260619 12:28:35.743216 27126 heartbeater.cc:347] Connected to a master server at 127.6.231.254:44327
I20260619 12:28:35.743494 27126 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:35.744051 27126 heartbeater.cc:511] Master 127.6.231.254:44327 requested a full tablet report, sending...
I20260619 12:28:35.745263 26807 ts_manager.cc:194] Registered new tserver with Master: 8dc809a2ec53461892463a0f4b13e6d6 (127.6.231.194:35427)
I20260619 12:28:35.746052 26807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.231.194:40113
W20260619 12:28:35.851399 27129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:35.851691 27129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:35.851733 27129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:35.855814 27129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:35.855883 27129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:35.855940 27129 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.6.231.195
I20260619 12:28:35.860814 27129 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:43705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.6.231.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.6.231.195
--webserver_port=0
--tserver_master_addrs=127.6.231.254:44327
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.6.231.195
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:35.862073 27129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:35.863425 27129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:35.873417 27135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:35.873474 27134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:35.873569 27129 server_base.cc:1034] running on GCE node
W20260619 12:28:35.873462 27137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:35.874161 27129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:35.874930 27129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:35.876104 27129 hybrid_clock.cc:648] HybridClock initialized: now 1781872115876087 us; error 26 us; skew 500 ppm
I20260619 12:28:35.878481 27129 webserver.cc:460] Webserver started at http://127.6.231.195:37693/ using document root <none> and password file <none>
I20260619 12:28:35.879185 27129 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:35.879257 27129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:35.879510 27129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:35.882063 27129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/data/instance:
uuid: "17c95e827ffe49f8a6273558606a5dc8"
format_stamp: "Formatted at 2026-06-19 12:28:35 on dist-test-slave-kr9h"
server_key: "f528e929002237d4f0ffd529c8e42e0d"
server_key_iv: "d8dbd0c32cd4c157637efe12db6fda6a"
server_key_version: "encryptionkey@0"
I20260619 12:28:35.882666 27129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance:
uuid: "17c95e827ffe49f8a6273558606a5dc8"
format_stamp: "Formatted at 2026-06-19 12:28:35 on dist-test-slave-kr9h"
server_key: "f528e929002237d4f0ffd529c8e42e0d"
server_key_iv: "d8dbd0c32cd4c157637efe12db6fda6a"
server_key_version: "encryptionkey@0"
I20260619 12:28:35.886786 27129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260619 12:28:35.889516 27143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:35.890884 27129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 12:28:35.890990 27129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/wal
uuid: "17c95e827ffe49f8a6273558606a5dc8"
format_stamp: "Formatted at 2026-06-19 12:28:35 on dist-test-slave-kr9h"
server_key: "f528e929002237d4f0ffd529c8e42e0d"
server_key_iv: "d8dbd0c32cd4c157637efe12db6fda6a"
server_key_version: "encryptionkey@0"
I20260619 12:28:35.891089 27129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:35.906203 27129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:35.906908 27129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:35.907068 27129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:35.907657 27129 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260619 12:28:35.908627 27129 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260619 12:28:35.908677 27129 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:35.908721 27129 ts_tablet_manager.cc:610] Registered 0 tablets
I20260619 12:28:35.908739 27129 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:35.919785 27129 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.195:38871
I20260619 12:28:35.919795 27256 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.195:38871 every 8 connection(s)
I20260619 12:28:35.920900 27129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/data/info.pb
I20260619 12:28:35.930605 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.922841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.195:36597 (local address 127.6.231.254:44327)
0619 12:28:35.922994 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:35.922998 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:35.923988 (+ 990us) server_negotiation.cc:407] Connection header received
0619 12:28:35.925602 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:35.925605 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:35.925656 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:35.925739 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:35.926749 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.927281 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.928469 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.928678 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.928762 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.930105 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:35.930109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:35.930112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:35.930128 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:35.930150 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:35.930159 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:35.930231 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:35.930410 (+ 179us) server_negotiation.cc:299] Negotiation successful
0619 12:28:35.930470 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 12:28:35.930972 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 27129
I20260619 12:28:35.931116 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/wal/instance
I20260619 12:28:35.931437 7071 external_mini_cluster.cc:1428] Setting key df02c3032a081dfedad5ff03e2ce0427
I20260619 12:28:35.931833 27259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.923152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:44327 (local address 127.6.231.195:36597)
0619 12:28:35.923832 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:35.923875 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:35.925310 (+ 1435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:35.925919 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:35.925926 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:35.926022 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:28:35.926606 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:35.926620 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.927398 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.927401 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:35.928363 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:35.928372 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.928826 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.928829 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:35.928982 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.929752 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0619 12:28:35.929754 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:28:35.929797 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0619 12:28:35.929869 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:28:35.930267 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:28:35.930275 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:28:35.930279 (+ 4us) client_negotiation.cc:769] Sending connection context
0619 12:28:35.930389 (+ 110us) client_negotiation.cc:240] Negotiation successful
0619 12:28:35.930631 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":107,"threads_started":1}
I20260619 12:28:35.933176 27257 heartbeater.cc:347] Connected to a master server at 127.6.231.254:44327
I20260619 12:28:35.933454 27257 heartbeater.cc:464] Registering TS with master...
I20260619 12:28:35.934067 27257 heartbeater.cc:511] Master 127.6.231.254:44327 requested a full tablet report, sending...
I20260619 12:28:35.935098 26807 ts_manager.cc:194] Registered new tserver with Master: 17c95e827ffe49f8a6273558606a5dc8 (127.6.231.195:38871)
I20260619 12:28:35.935724 26807 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.6.231.195:36597
I20260619 12:28:35.946007 7071 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260619 12:28:35.952360 26846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.948469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53176 (local address 127.6.231.254:44327)
0619 12:28:35.948652 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:35.948655 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:35.948764 (+ 109us) server_negotiation.cc:407] Connection header received
0619 12:28:35.948854 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:35.948856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:35.948926 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:35.949010 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:35.949393 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.950081 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.950952 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.951159 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.951242 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.951991 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:35.951994 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:35.951996 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:35.952007 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:35.952024 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:35.952032 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:35.952090 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:35.952190 (+ 100us) server_negotiation.cc:299] Negotiation successful
0619 12:28:35.952246 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260619 12:28:35.955238 26807 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53176:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260619 12:28:35.958393 26807 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260619 12:28:35.973027 27270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.967197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:39493 (local address 127.0.0.1:45596)
0619 12:28:35.967749 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:35.967765 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:35.967847 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:35.968570 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:35.968579 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:35.968604 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:35.968776 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:35.968783 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.970387 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.970390 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:35.971473 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:35.971482 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.972414 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.972418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:35.972523 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.972568 (+ 45us) client_negotiation.cc:769] Sending connection context
0619 12:28:35.972704 (+ 136us) client_negotiation.cc:240] Negotiation successful
0619 12:28:35.972850 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"mutex_wait_us":34,"thread_start_us":112,"threads_started":1}
I20260619 12:28:35.973057 27272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.967641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45596 (local address 127.6.231.193:39493)
0619 12:28:35.968054 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:35.968066 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:35.968092 (+ 26us) server_negotiation.cc:407] Connection header received
0619 12:28:35.968174 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:35.968179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:35.968388 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:35.968939 (+ 551us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:35.969089 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.970254 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.971595 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.972287 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.972395 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.972591 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:35.972751 (+ 160us) server_negotiation.cc:299] Negotiation successful
0619 12:28:35.972863 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":96,"threads_started":1}
I20260619 12:28:35.973217 27268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.966853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:38871 (local address 127.0.0.1:60064)
0619 12:28:35.967212 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:35.967269 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:35.967400 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:35.968613 (+ 1213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:35.968616 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:35.968636 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:35.968779 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:35.968785 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.970349 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.970352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:35.971929 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:35.971940 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.972858 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.972861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:35.972943 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.972963 (+ 20us) client_negotiation.cc:769] Sending connection context
0619 12:28:35.973008 (+ 45us) client_negotiation.cc:240] Negotiation successful
0619 12:28:35.973078 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":439,"thread_start_us":80,"threads_started":1}
I20260619 12:28:35.973791 27271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.967577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:35427 (local address 127.0.0.1:39926)
0619 12:28:35.968194 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:35.968210 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:35.968298 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:35.968941 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:35.968943 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:35.968958 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:35.969027 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:35.969033 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.970042 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.970048 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:35.971895 (+ 1847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:35.971903 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.973397 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.973401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:35.973510 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.973534 (+ 24us) client_negotiation.cc:769] Sending connection context
0619 12:28:35.973598 (+ 64us) client_negotiation.cc:240] Negotiation successful
0619 12:28:35.973666 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":432,"thread_start_us":103,"threads_started":1}
I20260619 12:28:35.974294 27269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.967039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60064 (local address 127.6.231.195:38871)
0619 12:28:35.968239 (+ 1200us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:35.968252 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:35.968272 (+ 20us) server_negotiation.cc:407] Connection header received
0619 12:28:35.968336 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:35.968339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:35.968489 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:35.968637 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:35.968909 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.970205 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.972062 (+ 1857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.972733 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.973494 (+ 761us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.973669 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:35.973719 (+ 50us) server_negotiation.cc:299] Negotiation successful
0619 12:28:35.973817 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1102,"thread_start_us":177,"threads_started":1}
I20260619 12:28:35.974480 27273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:35.967743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39926 (local address 127.6.231.194:35427)
0619 12:28:35.968544 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:35.968555 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:35.968576 (+ 21us) server_negotiation.cc:407] Connection header received
0619 12:28:35.968646 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:35.968650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:35.968827 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:35.968970 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:35.969104 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.969932 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.972016 (+ 2084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:35.972885 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:35.972990 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:35.974087 (+ 1097us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:35.974154 (+ 67us) server_negotiation.cc:299] Negotiation successful
0619 12:28:35.974249 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":699,"thread_start_us":90,"threads_started":1}
I20260619 12:28:35.976425 26929 tablet_service.cc:1467] Processing CreateTablet for tablet 3bb1ab6055b6414ba032ec696217aed4 (DEFAULT_TABLE table=test-table [id=8ed891c33d264066981c50ec8a36afb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:35.976922 27178 tablet_service.cc:1467] Processing CreateTablet for tablet 3bb1ab6055b6414ba032ec696217aed4 (DEFAULT_TABLE table=test-table [id=8ed891c33d264066981c50ec8a36afb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:35.977654 26929 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bb1ab6055b6414ba032ec696217aed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:35.977947 27178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bb1ab6055b6414ba032ec696217aed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:35.978932 27060 tablet_service.cc:1467] Processing CreateTablet for tablet 3bb1ab6055b6414ba032ec696217aed4 (DEFAULT_TABLE table=test-table [id=8ed891c33d264066981c50ec8a36afb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260619 12:28:35.979804 27060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3bb1ab6055b6414ba032ec696217aed4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:35.985220 27274 tablet_bootstrap.cc:492] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8: Bootstrap starting.
I20260619 12:28:35.985831 27275 tablet_bootstrap.cc:492] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e: Bootstrap starting.
I20260619 12:28:35.987375 27276 tablet_bootstrap.cc:492] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6: Bootstrap starting.
I20260619 12:28:35.987838 27275 tablet_bootstrap.cc:654] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:35.987838 27274 tablet_bootstrap.cc:654] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:35.988592 27274 log.cc:824] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:35.988668 27275 log.cc:824] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:35.989368 27276 tablet_bootstrap.cc:654] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:35.990197 27276 log.cc:824] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:35.990808 27275 tablet_bootstrap.cc:492] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e: No bootstrap required, opened a new log
I20260619 12:28:35.991014 27275 ts_tablet_manager.cc:1397] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260619 12:28:35.991065 27274 tablet_bootstrap.cc:492] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8: No bootstrap required, opened a new log
I20260619 12:28:35.991250 27274 ts_tablet_manager.cc:1397] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260619 12:28:35.992182 27276 tablet_bootstrap.cc:492] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6: No bootstrap required, opened a new log
I20260619 12:28:35.992357 27276 ts_tablet_manager.cc:1397] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.002s
I20260619 12:28:35.994738 27275 raft_consensus.cc:348] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17c95e827ffe49f8a6273558606a5dc8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 38871 } } peers { permanent_uuid: "8dc809a2ec53461892463a0f4b13e6d6" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35427 } } peers { permanent_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 39493 } }
I20260619 12:28:35.995086 27275 raft_consensus.cc:374] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:35.995142 27275 raft_consensus.cc:729] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e986f71d9ea49b799d9fd1fa222ea8e, State: Initialized, Role: FOLLOWER
I20260619 12:28:35.995620 27275 consensus_queue.cc:260] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17c95e827ffe49f8a6273558606a5dc8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 38871 } } peers { permanent_uuid: "8dc809a2ec53461892463a0f4b13e6d6" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35427 } } peers { permanent_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 39493 } }
I20260619 12:28:35.995601 27274 raft_consensus.cc:348] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17c95e827ffe49f8a6273558606a5dc8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 38871 } } peers { permanent_uuid: "8dc809a2ec53461892463a0f4b13e6d6" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35427 } } peers { permanent_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 39493 } }
I20260619 12:28:35.995849 27274 raft_consensus.cc:374] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:35.995903 27274 raft_consensus.cc:729] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 17c95e827ffe49f8a6273558606a5dc8, State: Initialized, Role: FOLLOWER
I20260619 12:28:35.996326 27274 consensus_queue.cc:260] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17c95e827ffe49f8a6273558606a5dc8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 38871 } } peers { permanent_uuid: "8dc809a2ec53461892463a0f4b13e6d6" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35427 } } peers { permanent_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 39493 } }
I20260619 12:28:35.996423 27276 raft_consensus.cc:348] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17c95e827ffe49f8a6273558606a5dc8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 38871 } } peers { permanent_uuid: "8dc809a2ec53461892463a0f4b13e6d6" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35427 } } peers { permanent_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 39493 } }
I20260619 12:28:35.996624 27275 ts_tablet_manager.cc:1428] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260619 12:28:35.996666 27276 raft_consensus.cc:374] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:35.996714 27276 raft_consensus.cc:729] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8dc809a2ec53461892463a0f4b13e6d6, State: Initialized, Role: FOLLOWER
I20260619 12:28:35.996980 26995 heartbeater.cc:503] Master 127.6.231.254:44327 was elected leader, sending a full tablet report...
I20260619 12:28:35.997251 27274 ts_tablet_manager.cc:1428] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260619 12:28:35.997601 27257 heartbeater.cc:503] Master 127.6.231.254:44327 was elected leader, sending a full tablet report...
I20260619 12:28:35.997781 27276 consensus_queue.cc:260] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17c95e827ffe49f8a6273558606a5dc8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 38871 } } peers { permanent_uuid: "8dc809a2ec53461892463a0f4b13e6d6" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35427 } } peers { permanent_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 39493 } }
I20260619 12:28:35.998510 27126 heartbeater.cc:503] Master 127.6.231.254:44327 was elected leader, sending a full tablet report...
I20260619 12:28:35.998685 27276 ts_tablet_manager.cc:1428] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6: Time spent starting tablet: real 0.006s user 0.003s sys 0.003s
W20260619 12:28:36.036154 26996 tablet.cc:2369] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260619 12:28:36.054137 27280 raft_consensus.cc:482] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260619 12:28:36.054381 27280 raft_consensus.cc:504] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17c95e827ffe49f8a6273558606a5dc8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 38871 } } peers { permanent_uuid: "8dc809a2ec53461892463a0f4b13e6d6" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35427 } } peers { permanent_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 39493 } }
I20260619 12:28:36.055596 27280 leader_election.cc:288] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 17c95e827ffe49f8a6273558606a5dc8 (127.6.231.195:38871), 8dc809a2ec53461892463a0f4b13e6d6 (127.6.231.194:35427)
I20260619 12:28:36.060226 27269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:36.056036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:50761 (local address 127.6.231.195:38871)
0619 12:28:36.056169 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:36.056173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:36.056455 (+ 282us) server_negotiation.cc:407] Connection header received
0619 12:28:36.056617 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:36.056622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:36.056687 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:36.056828 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:36.057277 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:36.057916 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:36.059126 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:36.059567 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:36.059666 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:36.059704 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:36.060008 (+ 304us) server_negotiation.cc:299] Negotiation successful
0619 12:28:36.060100 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:28:36.060238 27283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:36.055930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.195:38871 (local address 127.6.231.193:50761)
0619 12:28:36.056348 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:36.056363 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:36.056482 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:36.056848 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:36.056850 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:36.056870 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:36.057065 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:36.057073 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:36.058032 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:36.058035 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:36.058992 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:36.059001 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:36.059727 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:36.059731 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:36.059858 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:36.059895 (+ 37us) client_negotiation.cc:769] Sending connection context
0619 12:28:36.059972 (+ 77us) client_negotiation.cc:240] Negotiation successful
0619 12:28:36.060040 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"mutex_wait_us":35,"spinlock_wait_cycles":6912,"thread_start_us":161,"threads_started":1}
I20260619 12:28:36.060343 27273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:36.056567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.6.231.193:39279 (local address 127.6.231.194:35427)
0619 12:28:36.056719 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:36.056723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:36.056927 (+ 204us) server_negotiation.cc:407] Connection header received
0619 12:28:36.057063 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:36.057066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:36.057128 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:36.057208 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0619 12:28:36.057565 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:36.058121 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:36.059263 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:36.059750 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:36.059875 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:36.059920 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:36.060139 (+ 219us) server_negotiation.cc:299] Negotiation successful
0619 12:28:36.060212 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260619 12:28:36.060490 27284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:36.056410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.194:35427 (local address 127.6.231.193:39279)
0619 12:28:36.056822 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:36.056836 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:28:36.056947 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:28:36.057225 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:28:36.057230 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:28:36.057258 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0619 12:28:36.057367 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:36.057373 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:36.058249 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:36.058252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:36.059113 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:28:36.059123 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:28:36.059873 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:28:36.059876 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:28:36.059992 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:36.060017 (+ 25us) client_negotiation.cc:769] Sending connection context
0619 12:28:36.060090 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 12:28:36.060152 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":19,"thread_start_us":103,"threads_started":1}
I20260619 12:28:36.061124 27080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb1ab6055b6414ba032ec696217aed4" candidate_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc809a2ec53461892463a0f4b13e6d6" is_pre_election: true
I20260619 12:28:36.061131 27208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb1ab6055b6414ba032ec696217aed4" candidate_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17c95e827ffe49f8a6273558606a5dc8" is_pre_election: true
I20260619 12:28:36.061475 27080 raft_consensus.cc:2454] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e986f71d9ea49b799d9fd1fa222ea8e in term 0.
I20260619 12:28:36.061520 27208 raft_consensus.cc:2454] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4e986f71d9ea49b799d9fd1fa222ea8e in term 0.
I20260619 12:28:36.061988 26884 leader_election.cc:302] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e986f71d9ea49b799d9fd1fa222ea8e, 8dc809a2ec53461892463a0f4b13e6d6; no voters:
I20260619 12:28:36.062275 27280 raft_consensus.cc:2781] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260619 12:28:36.062366 27280 raft_consensus.cc:482] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260619 12:28:36.062412 27280 raft_consensus.cc:3037] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:36.063434 27280 raft_consensus.cc:504] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17c95e827ffe49f8a6273558606a5dc8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 38871 } } peers { permanent_uuid: "8dc809a2ec53461892463a0f4b13e6d6" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35427 } } peers { permanent_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 39493 } }
I20260619 12:28:36.063784 27280 leader_election.cc:288] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [CANDIDATE]: Term 1 election: Requested vote from peers 17c95e827ffe49f8a6273558606a5dc8 (127.6.231.195:38871), 8dc809a2ec53461892463a0f4b13e6d6 (127.6.231.194:35427)
I20260619 12:28:36.064131 27208 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb1ab6055b6414ba032ec696217aed4" candidate_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "17c95e827ffe49f8a6273558606a5dc8"
I20260619 12:28:36.064196 27080 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3bb1ab6055b6414ba032ec696217aed4" candidate_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8dc809a2ec53461892463a0f4b13e6d6"
I20260619 12:28:36.064335 27208 raft_consensus.cc:3037] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:36.064348 27080 raft_consensus.cc:3037] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:36.065671 27080 raft_consensus.cc:2454] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e986f71d9ea49b799d9fd1fa222ea8e in term 1.
I20260619 12:28:36.065740 27208 raft_consensus.cc:2454] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4e986f71d9ea49b799d9fd1fa222ea8e in term 1.
I20260619 12:28:36.066022 26884 leader_election.cc:302] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e986f71d9ea49b799d9fd1fa222ea8e, 8dc809a2ec53461892463a0f4b13e6d6; no voters:
I20260619 12:28:36.066306 27280 raft_consensus.cc:2781] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:36.066529 27280 raft_consensus.cc:686] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [term 1 LEADER]: Becoming Leader. State: Replica: 4e986f71d9ea49b799d9fd1fa222ea8e, State: Running, Role: LEADER
I20260619 12:28:36.066850 27280 consensus_queue.cc:237] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17c95e827ffe49f8a6273558606a5dc8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 38871 } } peers { permanent_uuid: "8dc809a2ec53461892463a0f4b13e6d6" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35427 } } peers { permanent_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 39493 } }
I20260619 12:28:36.070101 26807 catalog_manager.cc:5495] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e reported cstate change: term changed from 0 to 1, leader changed from <none> to 4e986f71d9ea49b799d9fd1fa222ea8e (127.6.231.193). New cstate: current_term: 1 leader_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "17c95e827ffe49f8a6273558606a5dc8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 38871 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8dc809a2ec53461892463a0f4b13e6d6" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4e986f71d9ea49b799d9fd1fa222ea8e" member_type: VOTER last_known_addr { host: "127.6.231.193" port: 39493 } health_report { overall_health: HEALTHY } } }
I20260619 12:28:36.099170 27272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:36.095023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45602 (local address 127.6.231.193:39493)
0619 12:28:36.095209 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:36.095213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:36.095230 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:28:36.095443 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:36.095446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:36.095511 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:36.095637 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:28:36.095992 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:36.096644 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:36.097578 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:36.097804 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:36.097889 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:36.098220 (+ 331us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:28:36.098675 (+ 455us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:28:36.098770 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:36.098889 (+ 119us) server_negotiation.cc:299] Negotiation successful
0619 12:28:36.098945 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260619 12:28:36.105571 27208 raft_consensus.cc:1261] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8 [term 1 FOLLOWER]: Refusing update from remote peer 4e986f71d9ea49b799d9fd1fa222ea8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:28:36.105571 27080 raft_consensus.cc:1261] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6 [term 1 FOLLOWER]: Refusing update from remote peer 4e986f71d9ea49b799d9fd1fa222ea8e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:28:36.106338 27280 consensus_queue.cc:1035] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "17c95e827ffe49f8a6273558606a5dc8" member_type: VOTER last_known_addr { host: "127.6.231.195" port: 38871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:28:36.106534 27285 consensus_queue.cc:1035] T 3bb1ab6055b6414ba032ec696217aed4 P 4e986f71d9ea49b799d9fd1fa222ea8e [LEADER]: Connected to new peer: Peer: permanent_uuid: "8dc809a2ec53461892463a0f4b13e6d6" member_type: VOTER last_known_addr { host: "127.6.231.194" port: 35427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:28:36.114315 27290 mvcc.cc:205] Tried to move back new op lower bound from 7298548187560669184 to 7298548187412504576. Current Snapshot: MvccSnapshot[applied={T|T < 7298548187560669184}]
I20260619 12:28:36.114600 27288 mvcc.cc:205] Tried to move back new op lower bound from 7298548187560669184 to 7298548187412504576. Current Snapshot: MvccSnapshot[applied={T|T < 7298548187560669184}]
I20260619 12:28:36.124179 26807 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:28:36.124387 26807 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53176:
table { table_name: "test-table" } modify_external_catalogs: true
I20260619 12:28:36.127251 26807 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2e6bd7797e274d94846ee1adc10fb92f: Sending DeleteTablet for 3 replicas of tablet 3bb1ab6055b6414ba032ec696217aed4
I20260619 12:28:36.128319 27178 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bb1ab6055b6414ba032ec696217aed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:36 UTC) from {username='slave'} at 127.0.0.1:60064
I20260619 12:28:36.128450 27060 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bb1ab6055b6414ba032ec696217aed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:36 UTC) from {username='slave'} at 127.0.0.1:39926
I20260619 12:28:36.128940 27299 tablet_replica.cc:330] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6: stopping tablet replica
I20260619 12:28:36.128959 26929 tablet_service.cc:1514] Processing DeleteTablet for tablet 3bb1ab6055b6414ba032ec696217aed4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-06-19 12:28:36 UTC) from {username='slave'} at 127.0.0.1:45596
I20260619 12:28:36.129194 27298 tablet_replica.cc:330] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8: stopping tablet replica
I20260619 12:28:36.129518 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 26867
I20260619 12:28:36.130424 27299 raft_consensus.cc:2229] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:28:36.130769 27299 raft_consensus.cc:2258] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:28:36.129580 27298 raft_consensus.cc:2229] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260619 12:28:36.131275 27298 raft_consensus.cc:2258] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260619 12:28:36.132588 27298 ts_tablet_manager.cc:1905] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:28:36.132776 27299 ts_tablet_manager.cc:1905] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260619 12:28:36.134935 27298 ts_tablet_manager.cc:1918] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:28:36.135047 27298 log.cc:1196] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-2/wal/wals/3bb1ab6055b6414ba032ec696217aed4
I20260619 12:28:36.135332 27298 ts_tablet_manager.cc:1939] T 3bb1ab6055b6414ba032ec696217aed4 P 17c95e827ffe49f8a6273558606a5dc8: Deleting consensus metadata
I20260619 12:28:36.136171 27299 ts_tablet_manager.cc:1918] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260619 12:28:36.136270 27299 log.cc:1196] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6: Deleting WAL directory at /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1781871857195028-7071-0/minicluster-data/ts-1/wal/wals/3bb1ab6055b6414ba032ec696217aed4
I20260619 12:28:36.136507 26793 catalog_manager.cc:4841] TS 17c95e827ffe49f8a6273558606a5dc8 (127.6.231.195:38871): tablet 3bb1ab6055b6414ba032ec696217aed4 (table test-table [id=8ed891c33d264066981c50ec8a36afb3]) successfully deleted
I20260619 12:28:36.136587 27299 ts_tablet_manager.cc:1939] T 3bb1ab6055b6414ba032ec696217aed4 P 8dc809a2ec53461892463a0f4b13e6d6: Deleting consensus metadata
I20260619 12:28:36.137912 26793 catalog_manager.cc:4841] TS 8dc809a2ec53461892463a0f4b13e6d6 (127.6.231.194:35427): tablet 3bb1ab6055b6414ba032ec696217aed4 (table test-table [id=8ed891c33d264066981c50ec8a36afb3]) successfully deleted
I20260619 12:28:36.137946 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 26998
W20260619 12:28:36.138036 26792 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.6.231.193:39493 (error 108)
I20260619 12:28:36.142401 27271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:36.142051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.193:39493 (local address 127.0.0.1:45614)
0619 12:28:36.142182 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 12:28:36.142314 (+ 132us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.193:39493: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 12:28:36.142674 26792 catalog_manager.cc:4584] TS 4e986f71d9ea49b799d9fd1fa222ea8e (127.6.231.193:39493): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3bb1ab6055b6414ba032ec696217aed4: Network error: Client connection negotiation failed: client connection to 127.6.231.193:39493: connect: Connection refused (error 111)
I20260619 12:28:36.143503 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 27129
I20260619 12:28:36.149106 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 26776
2026-06-19T12:28:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1093 ms)
[----------] 6 tests from EncryptionPolicyTest (10801 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:36 dist-test-slave-kr9h krb5kdc[27304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:36 dist-test-slave-kr9h krb5kdc[27304](info): set up 2 sockets
Jun 19 12:28:36 dist-test-slave-kr9h krb5kdc[27304](info): commencing operation
krb5kdc: starting...
W20260619 12:28:37.209769 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:37 dist-test-slave-kr9h krb5kdc[27304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:28:37.338950 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35107
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45733
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35107
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:37.460626 27317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:37.460976 27317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:37.461062 27317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:37.465217 27317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:37.465333 27317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:37.465374 27317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:37.465435 27317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:37.465500 27317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:37.471125 27317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45733
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35107
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35107
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:37.472294 27317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:37.473410 27317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:37.482667 27322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:37.482710 27325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:37.482800 27317 server_base.cc:1034] running on GCE node
W20260619 12:28:37.482671 27323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:37.483603 27317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:37.484906 27317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:37.486131 27317 hybrid_clock.cc:648] HybridClock initialized: now 1781872117485961 us; error 188 us; skew 500 ppm
Jun 19 12:28:37 dist-test-slave-kr9h krb5kdc[27304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872117, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:37.489225 27317 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:28:37.490336 27317 webserver.cc:460] Webserver started at http://127.0.0.1:34653/ using document root <none> and password file <none>
I20260619 12:28:37.490971 27317 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:37.491071 27317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:37.491315 27317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:37.493561 27317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "1f20363fdce842c2994b7a782c87648a"
format_stamp: "Formatted at 2026-06-19 12:28:37 on dist-test-slave-kr9h"
server_key: "e57f15ff4ae21d1e93927ee356e22e09"
server_key_iv: "971fcfa8915378735600e5cd66bcbf52"
server_key_version: "encryptionkey@0"
I20260619 12:28:37.494176 27317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "1f20363fdce842c2994b7a782c87648a"
format_stamp: "Formatted at 2026-06-19 12:28:37 on dist-test-slave-kr9h"
server_key: "e57f15ff4ae21d1e93927ee356e22e09"
server_key_iv: "971fcfa8915378735600e5cd66bcbf52"
server_key_version: "encryptionkey@0"
I20260619 12:28:37.498374 27317 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 12:28:37.501184 27332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:37.502749 27317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 12:28:37.502899 27317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "1f20363fdce842c2994b7a782c87648a"
format_stamp: "Formatted at 2026-06-19 12:28:37 on dist-test-slave-kr9h"
server_key: "e57f15ff4ae21d1e93927ee356e22e09"
server_key_iv: "971fcfa8915378735600e5cd66bcbf52"
server_key_version: "encryptionkey@0"
I20260619 12:28:37.503037 27317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:37.518940 27317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:37.522725 27317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:37.522946 27317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:37.530555 27317 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35107
I20260619 12:28:37.530558 27384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35107 every 8 connection(s)
I20260619 12:28:37.531582 27317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:37.534659 27385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:37.536406 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 27317
I20260619 12:28:37.536583 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:37.536898 7071 external_mini_cluster.cc:1428] Setting key cf553fd560c83734b9b854c97cc80423
I20260619 12:28:37.541560 27385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a: Bootstrap starting.
I20260619 12:28:37.543995 27385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:37.544822 27385 log.cc:824] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a: Log is configured to *not* fsync() on all Append() calls
Jun 19 12:28:37 dist-test-slave-kr9h krb5kdc[27304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 12:28:37.547108 27385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a: No bootstrap required, opened a new log
I20260619 12:28:37.551095 27385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f20363fdce842c2994b7a782c87648a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35107 } }
I20260619 12:28:37.551390 27385 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:37.551448 27385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f20363fdce842c2994b7a782c87648a, State: Initialized, Role: FOLLOWER
I20260619 12:28:37.551931 27385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f20363fdce842c2994b7a782c87648a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35107 } }
I20260619 12:28:37.552095 27385 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:37.552184 27385 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:37.552321 27385 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:37.552983 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:37.538529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39661 (local address 127.0.0.1:35107)
0619 12:28:37.539026 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:37.539042 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:37.539079 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:28:37.539816 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:37.539836 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:37.540198 (+ 362us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:37.540560 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:37.541376 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:37.542513 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:37.543426 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:37.543944 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:37.544121 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:37.546942 (+ 2821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:37.546948 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:37.546963 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:37.546992 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:37.549037 (+ 2045us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:37.549846 (+ 809us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:37.549852 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:37.549857 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:37.549987 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:37.550399 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:37.550405 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:37.550413 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:37.550874 (+ 461us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:37.551176 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:37.551453 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 12:28:37.551952 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":178,"threads_started":1}
I20260619 12:28:37.553488 27385 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f20363fdce842c2994b7a782c87648a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35107 } }
I20260619 12:28:37.553864 27385 leader_election.cc:302] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f20363fdce842c2994b7a782c87648a; no voters:
I20260619 12:28:37.554204 27385 leader_election.cc:288] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:37.554447 27390 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:37.554661 27390 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [term 1 LEADER]: Becoming Leader. State: Replica: 1f20363fdce842c2994b7a782c87648a, State: Running, Role: LEADER
I20260619 12:28:37.555011 27390 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f20363fdce842c2994b7a782c87648a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35107 } }
I20260619 12:28:37.555651 27385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:37.556704 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:37.557147 27391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f20363fdce842c2994b7a782c87648a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f20363fdce842c2994b7a782c87648a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35107 } } }
I20260619 12:28:37.557262 27391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:37.558254 27392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f20363fdce842c2994b7a782c87648a. Latest consensus state: current_term: 1 leader_uuid: "1f20363fdce842c2994b7a782c87648a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f20363fdce842c2994b7a782c87648a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35107 } } }
I20260619 12:28:37.558465 27392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:37.558573 27399 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:37.561913 27399 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:37.568794 27399 catalog_manager.cc:1347] Generated new cluster ID: 8fff1ab8af7a42d393a140dc604fe91d
I20260619 12:28:37.568876 27399 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:37.569032 27388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:37.559218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44643 (local address 127.0.0.1:35107)
0619 12:28:37.559360 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:37.559364 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:37.559529 (+ 165us) server_negotiation.cc:407] Connection header received
0619 12:28:37.559661 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:37.559663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:37.559704 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:37.559779 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:37.560667 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:37.561592 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:37.562891 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:37.563246 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:37.563358 (+ 112us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:37.564970 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:37.564974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:37.564976 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:37.565009 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:37.566607 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:37.567265 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:37.567269 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:37.567272 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:37.567376 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:37.567774 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:37.567779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:37.567781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:37.568031 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:37.568199 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:37.568790 (+ 591us) server_negotiation.cc:299] Negotiation successful
0619 12:28:37.568900 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:28:37.581000 27399 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:37.582093 27399 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:37.590116 27399 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f20363fdce842c2994b7a782c87648a: Generated new TSK 0
I20260619 12:28:37.590853 27399 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:37.599370 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 27317
W20260619 12:28:37.605724 27302 connection.cc:535] client connection to 127.0.0.1:35107 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35107): Connection reset by peer (error 104)
2026-06-19T12:28:37Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:37 dist-test-slave-kr9h krb5kdc[27416](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:37 dist-test-slave-kr9h krb5kdc[27416](info): set up 2 sockets
Jun 19 12:28:37 dist-test-slave-kr9h krb5kdc[27416](info): commencing operation
krb5kdc: starting...
W20260619 12:28:38.697662 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:38 dist-test-slave-kr9h krb5kdc[27416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872118, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:28:38.824304 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35817
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46495
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:38.938768 27429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:38.939057 27429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:38.939108 27429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:38.943356 27429 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:38.943431 27429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:38.943451 27429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:38.943468 27429 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:38.943532 27429 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:38.948948 27429 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46495
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35817
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:38.950233 27429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:38.951299 27429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:38.960955 27437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:38.961083 27429 server_base.cc:1034] running on GCE node
W20260619 12:28:38.960955 27434 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:38.960991 27435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:38.961966 27429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:38.963325 27429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:38.964586 27429 hybrid_clock.cc:648] HybridClock initialized: now 1781872118964564 us; error 35 us; skew 500 ppm
Jun 19 12:28:38 dist-test-slave-kr9h krb5kdc[27416](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872118, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:38.967643 27429 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:28:38.968643 27429 webserver.cc:460] Webserver started at http://127.0.0.1:41907/ using document root <none> and password file <none>
I20260619 12:28:38.969240 27429 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:38.969348 27429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:38.969614 27429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:38.971722 27429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "6912e0e6b25a4d8ab5926234f7bd1c1e"
format_stamp: "Formatted at 2026-06-19 12:28:38 on dist-test-slave-kr9h"
server_key: "85982990daafc38c4913b25c799bbeab"
server_key_iv: "c235e278723add5549a2ebf9ce599130"
server_key_version: "encryptionkey@0"
I20260619 12:28:38.972309 27429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "6912e0e6b25a4d8ab5926234f7bd1c1e"
format_stamp: "Formatted at 2026-06-19 12:28:38 on dist-test-slave-kr9h"
server_key: "85982990daafc38c4913b25c799bbeab"
server_key_iv: "c235e278723add5549a2ebf9ce599130"
server_key_version: "encryptionkey@0"
I20260619 12:28:38.976123 27429 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 12:28:38.978744 27444 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:38.980018 27429 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:28:38.980181 27429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "6912e0e6b25a4d8ab5926234f7bd1c1e"
format_stamp: "Formatted at 2026-06-19 12:28:38 on dist-test-slave-kr9h"
server_key: "85982990daafc38c4913b25c799bbeab"
server_key_iv: "c235e278723add5549a2ebf9ce599130"
server_key_version: "encryptionkey@0"
I20260619 12:28:38.980306 27429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:38.996788 27429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:39.000047 27429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:39.000278 27429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:39.007614 27429 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35817
I20260619 12:28:39.007639 27496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35817 every 8 connection(s)
I20260619 12:28:39.008608 27429 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:39.011125 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 27429
I20260619 12:28:39.011274 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:39.011612 7071 external_mini_cluster.cc:1428] Setting key afb203baf085e9a66339987653b19481
I20260619 12:28:39.011740 27497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:39.018172 27497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e: Bootstrap starting.
I20260619 12:28:39.020730 27497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:39.021519 27497 log.cc:824] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:39.021687 27500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:39.013039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59705 (local address 127.0.0.1:35817)
0619 12:28:39.013556 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:39.013573 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:39.013610 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:28:39.014536 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:39.014557 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:39.014932 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:39.015297 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:39.015708 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:39.016742 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:39.017729 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:39.018212 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:39.018398 (+ 186us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:39.019526 (+ 1128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:39.019534 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:39.019548 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:39.020069 (+ 521us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:39.020132 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:39.020159 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:39.020279 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:39.020556 (+ 277us) server_negotiation.cc:299] Negotiation successful
0619 12:28:39.020723 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":138,"threads_started":1}
I20260619 12:28:39.023687 27497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e: No bootstrap required, opened a new log
I20260619 12:28:39.024868 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:39.027892 27497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6912e0e6b25a4d8ab5926234f7bd1c1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35817 } }
I20260619 12:28:39.028177 27497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:39.028258 27497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6912e0e6b25a4d8ab5926234f7bd1c1e, State: Initialized, Role: FOLLOWER
I20260619 12:28:39.028760 27497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6912e0e6b25a4d8ab5926234f7bd1c1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35817 } }
I20260619 12:28:39.028936 27497 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:39.029007 27497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:39.029100 27497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:39.030290 27497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6912e0e6b25a4d8ab5926234f7bd1c1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35817 } }
I20260619 12:28:39.030651 27497 leader_election.cc:302] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6912e0e6b25a4d8ab5926234f7bd1c1e; no voters:
I20260619 12:28:39.031004 27497 leader_election.cc:288] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:39.031081 27507 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:39.031275 27507 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [term 1 LEADER]: Becoming Leader. State: Replica: 6912e0e6b25a4d8ab5926234f7bd1c1e, State: Running, Role: LEADER
I20260619 12:28:39.031569 27500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:39.027019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45975 (local address 127.0.0.1:35817)
0619 12:28:39.027135 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:39.027139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:39.027291 (+ 152us) server_negotiation.cc:407] Connection header received
0619 12:28:39.027432 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:39.027434 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:39.027476 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:39.027544 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:39.027846 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:39.028473 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:39.029617 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:39.029837 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:39.029919 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:39.030963 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:39.030966 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:39.030968 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:39.030981 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:39.031017 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:39.031028 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:39.031136 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:39.031402 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 12:28:39.031444 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260619 12:28:39.031580 27507 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6912e0e6b25a4d8ab5926234f7bd1c1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35817 } }
I20260619 12:28:39.032411 27497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:39.032982 27508 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6912e0e6b25a4d8ab5926234f7bd1c1e. Latest consensus state: current_term: 1 leader_uuid: "6912e0e6b25a4d8ab5926234f7bd1c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6912e0e6b25a4d8ab5926234f7bd1c1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35817 } } }
I20260619 12:28:39.033111 27508 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:39.033366 27509 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6912e0e6b25a4d8ab5926234f7bd1c1e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6912e0e6b25a4d8ab5926234f7bd1c1e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35817 } } }
I20260619 12:28:39.033519 27509 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:39.033666 27513 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:39.036688 27513 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:39.042718 27513 catalog_manager.cc:1347] Generated new cluster ID: 1d2c7d7d66b6400f8b89694d730b5472
I20260619 12:28:39.042799 27513 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:39.076025 27513 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:39.077355 27513 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:39.087251 27513 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6912e0e6b25a4d8ab5926234f7bd1c1e: Generated new TSK 0
I20260619 12:28:39.088104 27513 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:39.098078 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 27429
W20260619 12:28:39.103873 27414 connection.cc:535] client connection to 127.0.0.1:35817 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35817): Connection reset by peer (error 104)
2026-06-19T12:28:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1497 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:39 dist-test-slave-kr9h krb5kdc[27528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:39 dist-test-slave-kr9h krb5kdc[27528](info): set up 2 sockets
Jun 19 12:28:39 dist-test-slave-kr9h krb5kdc[27528](info): commencing operation
krb5kdc: starting...
W20260619 12:28:40.161784 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.043s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:40 dist-test-slave-kr9h krb5kdc[27528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:28:40.288056 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45525
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42265
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45525
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:40.399181 27541 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:40.399538 27541 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:40.399624 27541 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:40.403815 27541 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:40.403930 27541 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:40.403965 27541 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:40.404007 27541 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:40.404037 27541 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:40.409564 27541 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42265
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45525
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45525
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27541
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:40.410710 27541 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:40.411772 27541 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:40.420418 27547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:40.420420 27549 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:40.420718 27541 server_base.cc:1034] running on GCE node
W20260619 12:28:40.420418 27546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:40.421331 27541 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:40.422523 27541 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:40.423676 27541 hybrid_clock.cc:648] HybridClock initialized: now 1781872120423661 us; error 27 us; skew 500 ppm
Jun 19 12:28:40 dist-test-slave-kr9h krb5kdc[27528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872120, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:40.445457 27541 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:28:40.446637 27541 webserver.cc:460] Webserver started at http://127.0.0.1:42581/ using document root <none> and password file <none>
I20260619 12:28:40.447247 27541 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:40.447376 27541 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:40.447628 27541 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:40.449921 27541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "6a3e40a45f1e4076ba31327dd7570dc8"
format_stamp: "Formatted at 2026-06-19 12:28:40 on dist-test-slave-kr9h"
server_key: "461246e21b0bcaf568ce59dd6f6915b7"
server_key_iv: "31bad80577afe614a47d42251760f457"
server_key_version: "encryptionkey@0"
I20260619 12:28:40.450517 27541 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "6a3e40a45f1e4076ba31327dd7570dc8"
format_stamp: "Formatted at 2026-06-19 12:28:40 on dist-test-slave-kr9h"
server_key: "461246e21b0bcaf568ce59dd6f6915b7"
server_key_iv: "31bad80577afe614a47d42251760f457"
server_key_version: "encryptionkey@0"
I20260619 12:28:40.454499 27541 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260619 12:28:40.457103 27556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:40.458699 27541 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:28:40.458884 27541 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "6a3e40a45f1e4076ba31327dd7570dc8"
format_stamp: "Formatted at 2026-06-19 12:28:40 on dist-test-slave-kr9h"
server_key: "461246e21b0bcaf568ce59dd6f6915b7"
server_key_iv: "31bad80577afe614a47d42251760f457"
server_key_version: "encryptionkey@0"
I20260619 12:28:40.459025 27541 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:40.464056 27541 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:40.467463 27541 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:40.467670 27541 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:40.474915 27541 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45525
I20260619 12:28:40.475049 27608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45525 every 8 connection(s)
I20260619 12:28:40.475924 27541 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:40.479099 27609 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:40.484958 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 27541
I20260619 12:28:40.485126 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:40.485528 7071 external_mini_cluster.cc:1428] Setting key 6c386cc83121e0df42e473f745433f9d
I20260619 12:28:40.486243 27609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8: Bootstrap starting.
I20260619 12:28:40.488934 27609 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:40.489934 27609 log.cc:824] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:40.491159 27612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:40.487053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55455 (local address 127.0.0.1:45525)
0619 12:28:40.487585 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:40.487599 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:40.487635 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:28:40.488271 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:40.488289 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:40.488591 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:40.488935 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:40.489819 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:40.489823 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:40.489834 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:40.489845 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:40.489899 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:40.489918 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:40.490033 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:40.490345 (+ 312us) server_negotiation.cc:299] Negotiation successful
0619 12:28:40.490482 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":151,"threads_started":1}
I20260619 12:28:40.492041 27609 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8: No bootstrap required, opened a new log
I20260619 12:28:40.493886 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:40.497159 27609 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3e40a45f1e4076ba31327dd7570dc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45525 } }
I20260619 12:28:40.497556 27609 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:40.497630 27609 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a3e40a45f1e4076ba31327dd7570dc8, State: Initialized, Role: FOLLOWER
I20260619 12:28:40.498258 27609 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3e40a45f1e4076ba31327dd7570dc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45525 } }
I20260619 12:28:40.498471 27609 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:40.498643 27609 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:40.498662 27612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:40.496542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44603 (local address 127.0.0.1:45525)
0619 12:28:40.496665 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:40.496672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:40.496885 (+ 213us) server_negotiation.cc:407] Connection header received
0619 12:28:40.496995 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:40.496997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:40.497030 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:40.497103 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:40.498067 (+ 964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:40.498070 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:40.498072 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:40.498082 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:40.498107 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:40.498118 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:40.498195 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:40.498377 (+ 182us) server_negotiation.cc:299] Negotiation successful
0619 12:28:40.498451 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260619 12:28:40.498797 27609 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:40.500025 27609 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3e40a45f1e4076ba31327dd7570dc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45525 } }
I20260619 12:28:40.500408 27609 leader_election.cc:302] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a3e40a45f1e4076ba31327dd7570dc8; no voters:
I20260619 12:28:40.500774 27609 leader_election.cc:288] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:40.500947 27619 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:40.501183 27619 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [term 1 LEADER]: Becoming Leader. State: Replica: 6a3e40a45f1e4076ba31327dd7570dc8, State: Running, Role: LEADER
I20260619 12:28:40.501761 27619 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3e40a45f1e4076ba31327dd7570dc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45525 } }
I20260619 12:28:40.502226 27609 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:40.502981 27619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a3e40a45f1e4076ba31327dd7570dc8. Latest consensus state: current_term: 1 leader_uuid: "6a3e40a45f1e4076ba31327dd7570dc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3e40a45f1e4076ba31327dd7570dc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45525 } } }
I20260619 12:28:40.503171 27619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:40.503399 27620 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a3e40a45f1e4076ba31327dd7570dc8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a3e40a45f1e4076ba31327dd7570dc8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45525 } } }
I20260619 12:28:40.503523 27620 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8 [sys.catalog]: This master's current role is: LEADER
W20260619 12:28:40.507293 27634 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:28:40.507375 27634 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:28:40.507458 27628 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:40.508354 27628 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:40.514411 27628 catalog_manager.cc:1347] Generated new cluster ID: a0ada26542904181b56323477f23efc5
I20260619 12:28:40.514499 27628 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:40.524812 27628 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:40.525173 27628 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:40.542379 27628 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a3e40a45f1e4076ba31327dd7570dc8: Generated new TSK 0
I20260619 12:28:40.543223 27628 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:40.549562 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 27541
2026-06-19T12:28:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1450 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:40 dist-test-slave-kr9h krb5kdc[27640](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:40 dist-test-slave-kr9h krb5kdc[27640](info): set up 2 sockets
Jun 19 12:28:40 dist-test-slave-kr9h krb5kdc[27640](info): commencing operation
krb5kdc: starting...
W20260619 12:28:41.593743 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:41 dist-test-slave-kr9h krb5kdc[27640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:28:41.720564 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46803
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34405
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:41.831547 27653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:41.831835 27653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:41.831879 27653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:41.836022 27653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:41.836095 27653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:41.836117 27653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:41.836134 27653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:41.836150 27653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:41.842025 27653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34405
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:41.843205 27653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:41.844283 27653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:28:41.853505 27653 server_base.cc:1034] running on GCE node
W20260619 12:28:41.853411 27659 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:41.853477 27658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:41.853411 27661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:41.854252 27653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:41.855432 27653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:41.856604 27653 hybrid_clock.cc:648] HybridClock initialized: now 1781872121856570 us; error 47 us; skew 500 ppm
Jun 19 12:28:41 dist-test-slave-kr9h krb5kdc[27640](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872121, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:41.859462 27653 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:28:41.860379 27653 webserver.cc:460] Webserver started at http://127.0.0.1:38699/ using document root <none> and password file <none>
I20260619 12:28:41.860911 27653 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:41.860971 27653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:41.861148 27653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:41.863349 27653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "367e18e56d30428bb2a3e030f6e66286"
format_stamp: "Formatted at 2026-06-19 12:28:41 on dist-test-slave-kr9h"
server_key: "c46b3783a58536c2c2a17be02b3efb32"
server_key_iv: "856ed2c9fbd6e5ecf8f5416ca3c30d0d"
server_key_version: "encryptionkey@0"
I20260619 12:28:41.863883 27653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "367e18e56d30428bb2a3e030f6e66286"
format_stamp: "Formatted at 2026-06-19 12:28:41 on dist-test-slave-kr9h"
server_key: "c46b3783a58536c2c2a17be02b3efb32"
server_key_iv: "856ed2c9fbd6e5ecf8f5416ca3c30d0d"
server_key_version: "encryptionkey@0"
I20260619 12:28:41.867739 27653 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:28:41.870262 27668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:41.871551 27653 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:28:41.871721 27653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "367e18e56d30428bb2a3e030f6e66286"
format_stamp: "Formatted at 2026-06-19 12:28:41 on dist-test-slave-kr9h"
server_key: "c46b3783a58536c2c2a17be02b3efb32"
server_key_iv: "856ed2c9fbd6e5ecf8f5416ca3c30d0d"
server_key_version: "encryptionkey@0"
I20260619 12:28:41.871855 27653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:41.887705 27653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:41.890787 27653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:41.891000 27653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:41.898219 27653 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46803
I20260619 12:28:41.898273 27720 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46803 every 8 connection(s)
I20260619 12:28:41.899206 27653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:41.902055 27721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:41.907449 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 27653
I20260619 12:28:41.907529 27721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286: Bootstrap starting.
I20260619 12:28:41.907639 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:41.907951 7071 external_mini_cluster.cc:1428] Setting key ee411da98faf1ce8e88b51ca0114d118
I20260619 12:28:41.910210 27721 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:41.910943 27721 log.cc:824] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:41.912770 27721 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286: No bootstrap required, opened a new log
I20260619 12:28:41.916121 27721 raft_consensus.cc:348] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367e18e56d30428bb2a3e030f6e66286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46803 } }
I20260619 12:28:41.916361 27721 raft_consensus.cc:374] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:41.916440 27721 raft_consensus.cc:729] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 367e18e56d30428bb2a3e030f6e66286, State: Initialized, Role: FOLLOWER
Jun 19 12:28:41 dist-test-slave-kr9h krb5kdc[27640](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872121, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 12:28:41.916953 27721 consensus_queue.cc:260] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367e18e56d30428bb2a3e030f6e66286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46803 } }
I20260619 12:28:41.917137 27721 raft_consensus.cc:388] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:41.917225 27721 raft_consensus.cc:482] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:41.917321 27721 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:41.918253 27721 raft_consensus.cc:504] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367e18e56d30428bb2a3e030f6e66286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46803 } }
I20260619 12:28:41.918601 27721 leader_election.cc:302] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 367e18e56d30428bb2a3e030f6e66286; no voters:
I20260619 12:28:41.918942 27721 leader_election.cc:288] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:41.919018 27726 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:41.919225 27726 raft_consensus.cc:686] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [term 1 LEADER]: Becoming Leader. State: Replica: 367e18e56d30428bb2a3e030f6e66286, State: Running, Role: LEADER
I20260619 12:28:41.919579 27726 consensus_queue.cc:237] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367e18e56d30428bb2a3e030f6e66286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46803 } }
I20260619 12:28:41.920264 27721 sys_catalog.cc:564] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:41.921255 27728 sys_catalog.cc:455] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 367e18e56d30428bb2a3e030f6e66286. Latest consensus state: current_term: 1 leader_uuid: "367e18e56d30428bb2a3e030f6e66286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367e18e56d30428bb2a3e030f6e66286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46803 } } }
I20260619 12:28:41.921373 27728 sys_catalog.cc:458] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:41.921711 27727 sys_catalog.cc:455] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "367e18e56d30428bb2a3e030f6e66286" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "367e18e56d30428bb2a3e030f6e66286" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46803 } } }
I20260619 12:28:41.921767 27735 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:41.921801 27727 sys_catalog.cc:458] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:41.923998 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:41.909326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52211 (local address 127.0.0.1:46803)
0619 12:28:41.909805 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:41.909822 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:41.909858 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:28:41.910666 (+ 808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:41.910690 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:41.911058 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:41.911440 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:41.912251 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:41.913288 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:41.914204 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:41.914689 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:41.914861 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:41.917598 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:41.917611 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:41.917625 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:41.917652 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:41.919907 (+ 2255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:41.920372 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:41.920378 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:41.920384 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:41.920550 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:41.920908 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:41.920914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:41.920917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:41.921580 (+ 663us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:41.922525 (+ 945us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:41.922695 (+ 170us) server_negotiation.cc:299] Negotiation successful
0619 12:28:41.923015 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":146,"threads_started":1}
I20260619 12:28:41.925130 27735 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:41.927156 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:41.931430 27735 catalog_manager.cc:1347] Generated new cluster ID: a652b044cf2c41c2ad400a8bd3f22310
I20260619 12:28:41.931502 27735 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:41.938977 27724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:41.929672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53531 (local address 127.0.0.1:46803)
0619 12:28:41.929788 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:41.929792 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:41.930090 (+ 298us) server_negotiation.cc:407] Connection header received
0619 12:28:41.930131 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:41.930134 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:41.930179 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:41.930261 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:41.931269 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:41.932162 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:41.933309 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:41.933707 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:41.933819 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:41.935441 (+ 1622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:41.935445 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:41.935448 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:41.935477 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:41.937238 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:41.937746 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:41.937750 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:41.937752 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:41.937843 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:41.938108 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:41.938111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:41.938113 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:41.938331 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:41.938476 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:41.938682 (+ 206us) server_negotiation.cc:299] Negotiation successful
0619 12:28:41.938804 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:28:41.946877 27735 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:41.947815 27735 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:41.955763 27735 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 367e18e56d30428bb2a3e030f6e66286: Generated new TSK 0
I20260619 12:28:41.956588 27735 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:41.966193 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 27653
2026-06-19T12:28:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1417 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:42 dist-test-slave-kr9h krb5kdc[27752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:42 dist-test-slave-kr9h krb5kdc[27752](info): set up 2 sockets
Jun 19 12:28:42 dist-test-slave-kr9h krb5kdc[27752](info): commencing operation
krb5kdc: starting...
W20260619 12:28:43.005735 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.020s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:43 dist-test-slave-kr9h krb5kdc[27752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872123, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:28:43.134616 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37391
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37391
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:43.254539 27765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:43.254884 27765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:43.254983 27765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:43.259644 27765 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:43.259757 27765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:43.259794 27765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:43.259846 27765 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:43.259902 27765 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:43.265851 27765 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37391
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37391
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:43.267055 27765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:43.268172 27765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:43.277745 27770 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:43.278064 27765 server_base.cc:1034] running on GCE node
W20260619 12:28:43.277745 27771 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:43.277745 27773 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:43.278815 27765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:43.280328 27765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:43.281646 27765 hybrid_clock.cc:648] HybridClock initialized: now 1781872123281617 us; error 40 us; skew 500 ppm
Jun 19 12:28:43 dist-test-slave-kr9h krb5kdc[27752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872123, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:43.284644 27765 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:28:43.285686 27765 webserver.cc:460] Webserver started at http://127.0.0.1:43939/ using document root <none> and password file <none>
I20260619 12:28:43.286284 27765 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:43.286350 27765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:43.286540 27765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:43.288667 27765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "bbb65b78dd2a4f83806ec8607a4611ac"
format_stamp: "Formatted at 2026-06-19 12:28:43 on dist-test-slave-kr9h"
server_key: "b8003761146dfd0002d5902203f22b12"
server_key_iv: "bbacc914566baa4eb6b0e3bdb4770a49"
server_key_version: "encryptionkey@0"
I20260619 12:28:43.289207 27765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "bbb65b78dd2a4f83806ec8607a4611ac"
format_stamp: "Formatted at 2026-06-19 12:28:43 on dist-test-slave-kr9h"
server_key: "b8003761146dfd0002d5902203f22b12"
server_key_iv: "bbacc914566baa4eb6b0e3bdb4770a49"
server_key_version: "encryptionkey@0"
I20260619 12:28:43.293329 27765 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.000s
I20260619 12:28:43.296200 27780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:43.297765 27765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260619 12:28:43.297900 27765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "bbb65b78dd2a4f83806ec8607a4611ac"
format_stamp: "Formatted at 2026-06-19 12:28:43 on dist-test-slave-kr9h"
server_key: "b8003761146dfd0002d5902203f22b12"
server_key_iv: "bbacc914566baa4eb6b0e3bdb4770a49"
server_key_version: "encryptionkey@0"
I20260619 12:28:43.298017 27765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:43.327651 27765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:43.331527 27765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:43.331712 27765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:43.340528 27765 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37391
I20260619 12:28:43.340564 27832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37391 every 8 connection(s)
I20260619 12:28:43.341638 27765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:43.342235 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 27765
I20260619 12:28:43.342411 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:43.342718 7071 external_mini_cluster.cc:1428] Setting key 922a1d4b3e47d72a28ffba0829d80138
I20260619 12:28:43.345089 27833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:43.350934 27833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac: Bootstrap starting.
I20260619 12:28:43.352283 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:43.344215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54993 (local address 127.0.0.1:37391)
0619 12:28:43.344683 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:43.344698 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:43.344735 (+ 37us) server_negotiation.cc:407] Connection header received
0619 12:28:43.345577 (+ 842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:43.345597 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:43.345957 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:43.346333 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:43.346764 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:43.347841 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:43.348772 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:43.349272 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:43.349469 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:43.350443 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:43.350450 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:43.350463 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:43.350874 (+ 411us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:43.350932 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:43.350955 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:43.351089 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:43.351229 (+ 140us) server_negotiation.cc:299] Negotiation successful
0619 12:28:43.351471 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":152,"threads_started":1}
I20260619 12:28:43.353552 27833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:43.354373 27833 log.cc:824] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:43.355576 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:43.356597 27833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac: No bootstrap required, opened a new log
I20260619 12:28:43.360795 27833 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb65b78dd2a4f83806ec8607a4611ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37391 } }
I20260619 12:28:43.361119 27833 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:43.361191 27833 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbb65b78dd2a4f83806ec8607a4611ac, State: Initialized, Role: FOLLOWER
I20260619 12:28:43.361827 27833 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb65b78dd2a4f83806ec8607a4611ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37391 } }
I20260619 12:28:43.362054 27833 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:43.362228 27833 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:43.362270 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:43.357954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38075 (local address 127.0.0.1:37391)
0619 12:28:43.358099 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:43.358105 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:43.358170 (+ 65us) server_negotiation.cc:407] Connection header received
0619 12:28:43.358330 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:43.358334 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:43.358387 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:43.358561 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:43.358849 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:43.359396 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:43.360282 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:43.360534 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:43.360653 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:43.361629 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:43.361633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:43.361635 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:43.361650 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:43.361681 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:43.361695 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:43.361800 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:43.361963 (+ 163us) server_negotiation.cc:299] Negotiation successful
0619 12:28:43.362034 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:28:43.362433 27833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:43.363842 27833 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb65b78dd2a4f83806ec8607a4611ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37391 } }
I20260619 12:28:43.364280 27833 leader_election.cc:302] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbb65b78dd2a4f83806ec8607a4611ac; no voters:
I20260619 12:28:43.364693 27833 leader_election.cc:288] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:43.364925 27843 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:43.365237 27843 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [term 1 LEADER]: Becoming Leader. State: Replica: bbb65b78dd2a4f83806ec8607a4611ac, State: Running, Role: LEADER
I20260619 12:28:43.365700 27843 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb65b78dd2a4f83806ec8607a4611ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37391 } }
I20260619 12:28:43.366374 27833 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:43.367986 27845 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbb65b78dd2a4f83806ec8607a4611ac. Latest consensus state: current_term: 1 leader_uuid: "bbb65b78dd2a4f83806ec8607a4611ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb65b78dd2a4f83806ec8607a4611ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37391 } } }
I20260619 12:28:43.368181 27845 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:43.367980 27844 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbb65b78dd2a4f83806ec8607a4611ac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbb65b78dd2a4f83806ec8607a4611ac" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37391 } } }
I20260619 12:28:43.368275 27844 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:43.368705 27854 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:43.372721 27854 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:43.379120 27854 catalog_manager.cc:1347] Generated new cluster ID: 9a0e95720b1f43aea20335bdfb34c8e6
I20260619 12:28:43.379211 27854 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:43.427892 27854 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:43.428921 27854 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:43.437736 27854 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbb65b78dd2a4f83806ec8607a4611ac: Generated new TSK 0
I20260619 12:28:43.438547 27854 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:43.449373 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 27765
2026-06-19T12:28:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1483 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:43 dist-test-slave-kr9h krb5kdc[27864](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:43 dist-test-slave-kr9h krb5kdc[27864](info): set up 2 sockets
Jun 19 12:28:43 dist-test-slave-kr9h krb5kdc[27864](info): commencing operation
krb5kdc: starting...
W20260619 12:28:44.581890 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:44 dist-test-slave-kr9h krb5kdc[27864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:28:44.710150 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45747
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45747
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:44.824759 27877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:44.825085 27877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:44.825131 27877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:44.829335 27877 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:44.829425 27877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:44.829468 27877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:44.829485 27877 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:44.829500 27877 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:44.835017 27877 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45747
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27877
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:44.836161 27877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:44.837170 27877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:44.846478 27885 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:44.846604 27877 server_base.cc:1034] running on GCE node
W20260619 12:28:44.846478 27883 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:44.846520 27882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:44.847535 27877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:44.848817 27877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:44.850059 27877 hybrid_clock.cc:648] HybridClock initialized: now 1781872124850029 us; error 41 us; skew 500 ppm
Jun 19 12:28:44 dist-test-slave-kr9h krb5kdc[27864](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872124, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:44.853262 27877 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:28:44.854398 27877 webserver.cc:460] Webserver started at http://127.0.0.1:45237/ using document root <none> and password file <none>
I20260619 12:28:44.855037 27877 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:44.855135 27877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:44.855383 27877 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:44.857798 27877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "eb6e2547aa044f02abb60ae565dc1cb7"
format_stamp: "Formatted at 2026-06-19 12:28:44 on dist-test-slave-kr9h"
server_key: "76126983bda997bb01765874c3ea7dbc"
server_key_iv: "1473943f22446c9ba1db402238527fb6"
server_key_version: "encryptionkey@0"
I20260619 12:28:44.858397 27877 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "eb6e2547aa044f02abb60ae565dc1cb7"
format_stamp: "Formatted at 2026-06-19 12:28:44 on dist-test-slave-kr9h"
server_key: "76126983bda997bb01765874c3ea7dbc"
server_key_iv: "1473943f22446c9ba1db402238527fb6"
server_key_version: "encryptionkey@0"
I20260619 12:28:44.862433 27877 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260619 12:28:44.865125 27892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:44.866742 27877 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260619 12:28:44.866904 27877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "eb6e2547aa044f02abb60ae565dc1cb7"
format_stamp: "Formatted at 2026-06-19 12:28:44 on dist-test-slave-kr9h"
server_key: "76126983bda997bb01765874c3ea7dbc"
server_key_iv: "1473943f22446c9ba1db402238527fb6"
server_key_version: "encryptionkey@0"
I20260619 12:28:44.867040 27877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:44.872248 27877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:44.875842 27877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:44.876075 27877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:44.883466 27877 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45747
I20260619 12:28:44.883478 27944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45747 every 8 connection(s)
I20260619 12:28:44.884474 27877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:44.887017 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 27877
I20260619 12:28:44.887204 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:44.887552 7071 external_mini_cluster.cc:1428] Setting key 5c3843a99783bd912b5c725ee9c05796
I20260619 12:28:44.887610 27945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:44.893754 27948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:44.889071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37067 (local address 127.0.0.1:45747)
0619 12:28:44.889542 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:44.889561 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:44.889604 (+ 43us) server_negotiation.cc:407] Connection header received
0619 12:28:44.890582 (+ 978us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:44.890607 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:44.891008 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:44.891397 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:44.892210 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:44.892216 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:44.892232 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:44.892244 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:44.892311 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:44.892339 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:44.892494 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:44.892600 (+ 106us) server_negotiation.cc:299] Negotiation successful
0619 12:28:44.892767 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":121,"threads_started":1}
I20260619 12:28:44.895557 27945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7: Bootstrap starting.
I20260619 12:28:44.896970 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:44.898088 27945 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:44.898983 27945 log.cc:824] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:44.900990 27945 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7: No bootstrap required, opened a new log
I20260619 12:28:44.901038 27948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:44.899245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37663 (local address 127.0.0.1:45747)
0619 12:28:44.899400 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:44.899405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:44.899567 (+ 162us) server_negotiation.cc:407] Connection header received
0619 12:28:44.899737 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:44.899740 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:44.899782 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:44.899869 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:44.900639 (+ 770us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:44.900642 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:44.900645 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:44.900656 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:44.900692 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:44.900709 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:44.900808 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:44.900878 (+ 70us) server_negotiation.cc:299] Negotiation successful
0619 12:28:44.900920 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260619 12:28:44.904788 27945 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb6e2547aa044f02abb60ae565dc1cb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45747 } }
I20260619 12:28:44.905071 27945 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:44.905157 27945 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb6e2547aa044f02abb60ae565dc1cb7, State: Initialized, Role: FOLLOWER
I20260619 12:28:44.905683 27945 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb6e2547aa044f02abb60ae565dc1cb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45747 } }
I20260619 12:28:44.905841 27945 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:44.905930 27945 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:44.906033 27945 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:44.907043 27945 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb6e2547aa044f02abb60ae565dc1cb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45747 } }
I20260619 12:28:44.907426 27945 leader_election.cc:302] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb6e2547aa044f02abb60ae565dc1cb7; no voters:
I20260619 12:28:44.907876 27945 leader_election.cc:288] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:44.907951 27955 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:44.908169 27955 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [term 1 LEADER]: Becoming Leader. State: Replica: eb6e2547aa044f02abb60ae565dc1cb7, State: Running, Role: LEADER
I20260619 12:28:44.908525 27955 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb6e2547aa044f02abb60ae565dc1cb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45747 } }
I20260619 12:28:44.909266 27945 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:44.910357 27957 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb6e2547aa044f02abb60ae565dc1cb7. Latest consensus state: current_term: 1 leader_uuid: "eb6e2547aa044f02abb60ae565dc1cb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb6e2547aa044f02abb60ae565dc1cb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45747 } } }
I20260619 12:28:44.910509 27957 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:44.910806 27956 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb6e2547aa044f02abb60ae565dc1cb7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb6e2547aa044f02abb60ae565dc1cb7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45747 } } }
I20260619 12:28:44.910897 27956 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7 [sys.catalog]: This master's current role is: LEADER
W20260619 12:28:44.914697 27971 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:28:44.914817 27971 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:28:44.914889 27964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:44.915654 27964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:44.922060 27964 catalog_manager.cc:1347] Generated new cluster ID: 4bdf860ee14f42c0b903fe0c31257b53
I20260619 12:28:44.922166 27964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:44.930903 27964 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:44.931290 27964 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:44.939667 27964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb6e2547aa044f02abb60ae565dc1cb7: Generated new TSK 0
I20260619 12:28:44.940598 27964 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:44.951591 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 27877
2026-06-19T12:28:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1503 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:45 dist-test-slave-kr9h krb5kdc[27976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:45 dist-test-slave-kr9h krb5kdc[27976](info): set up 2 sockets
Jun 19 12:28:45 dist-test-slave-kr9h krb5kdc[27976](info): commencing operation
krb5kdc: starting...
W20260619 12:28:46.033941 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.061s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:46 dist-test-slave-kr9h krb5kdc[27976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:28:46.161082 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42475
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41179
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:46.286949 27989 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:46.287220 27989 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:46.287266 27989 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:46.291414 27989 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:46.291489 27989 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:46.291512 27989 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:46.291528 27989 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:46.291543 27989 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:46.297103 27989 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41179
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27989
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:46.298261 27989 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:46.299263 27989 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:46.308038 27997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:46.308048 27995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:46.308045 27994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:46.308471 27989 server_base.cc:1034] running on GCE node
I20260619 12:28:46.309005 27989 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:46.310160 27989 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:46.311355 27989 hybrid_clock.cc:648] HybridClock initialized: now 1781872126311336 us; error 32 us; skew 500 ppm
Jun 19 12:28:46 dist-test-slave-kr9h krb5kdc[27976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872126, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:46.314160 27989 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:28:46.315076 27989 webserver.cc:460] Webserver started at http://127.0.0.1:40513/ using document root <none> and password file <none>
I20260619 12:28:46.315640 27989 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:46.315701 27989 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:46.315877 27989 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:46.318091 27989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "b8807cf71ce944268d0e7d587eaf47f6"
format_stamp: "Formatted at 2026-06-19 12:28:46 on dist-test-slave-kr9h"
server_key: "dce9780ec53b009b6ef1602992633f93"
server_key_iv: "b089894072541b1366943fadd5f3b5a9"
server_key_version: "encryptionkey@0"
I20260619 12:28:46.318602 27989 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "b8807cf71ce944268d0e7d587eaf47f6"
format_stamp: "Formatted at 2026-06-19 12:28:46 on dist-test-slave-kr9h"
server_key: "dce9780ec53b009b6ef1602992633f93"
server_key_iv: "b089894072541b1366943fadd5f3b5a9"
server_key_version: "encryptionkey@0"
I20260619 12:28:46.322382 27989 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:46.324791 28004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:46.326105 27989 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:28:46.326239 27989 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "b8807cf71ce944268d0e7d587eaf47f6"
format_stamp: "Formatted at 2026-06-19 12:28:46 on dist-test-slave-kr9h"
server_key: "dce9780ec53b009b6ef1602992633f93"
server_key_iv: "b089894072541b1366943fadd5f3b5a9"
server_key_version: "encryptionkey@0"
I20260619 12:28:46.326346 27989 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:46.356289 27989 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:46.359895 27989 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:46.360114 27989 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:46.367628 27989 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42475
I20260619 12:28:46.367647 28056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42475 every 8 connection(s)
I20260619 12:28:46.368707 27989 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:46.371567 28057 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:46.377547 28057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6: Bootstrap starting.
I20260619 12:28:46.378367 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 27989
I20260619 12:28:46.378536 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:46.378831 7071 external_mini_cluster.cc:1428] Setting key f6c35224ef112ab144db4a03b84915b9
I20260619 12:28:46.380520 28057 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:46.381418 28057 log.cc:824] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:46.383715 28057 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6: No bootstrap required, opened a new log
I20260619 12:28:46.387800 28057 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8807cf71ce944268d0e7d587eaf47f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42475 } }
I20260619 12:28:46.388031 28057 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:46.388092 28057 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8807cf71ce944268d0e7d587eaf47f6, State: Initialized, Role: FOLLOWER
I20260619 12:28:46.388535 28057 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8807cf71ce944268d0e7d587eaf47f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42475 } }
I20260619 12:28:46.388655 28057 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
Jun 19 12:28:46 dist-test-slave-kr9h krb5kdc[27976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 12:28:46.388799 28057 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:46.388936 28057 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:46.390214 28057 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8807cf71ce944268d0e7d587eaf47f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42475 } }
I20260619 12:28:46.390630 28057 leader_election.cc:302] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8807cf71ce944268d0e7d587eaf47f6; no voters:
I20260619 12:28:46.391076 28057 leader_election.cc:288] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:46.391180 28062 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:46.391485 28062 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [term 1 LEADER]: Becoming Leader. State: Replica: b8807cf71ce944268d0e7d587eaf47f6, State: Running, Role: LEADER
I20260619 12:28:46.391865 28062 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8807cf71ce944268d0e7d587eaf47f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42475 } }
I20260619 12:28:46.392539 28057 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:46.393759 28064 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8807cf71ce944268d0e7d587eaf47f6. Latest consensus state: current_term: 1 leader_uuid: "b8807cf71ce944268d0e7d587eaf47f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8807cf71ce944268d0e7d587eaf47f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42475 } } }
I20260619 12:28:46.393882 28064 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:46.394250 28063 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8807cf71ce944268d0e7d587eaf47f6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8807cf71ce944268d0e7d587eaf47f6" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42475 } } }
I20260619 12:28:46.394428 28063 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:46.394752 28071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:46.396087 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:46.380052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:39865 (local address 127.0.0.1:42475)
0619 12:28:46.380553 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:46.380573 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:46.380617 (+ 44us) server_negotiation.cc:407] Connection header received
0619 12:28:46.381573 (+ 956us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:46.381599 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:46.382068 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:46.382450 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:46.383155 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:46.384634 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:46.385632 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:46.386401 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:46.386594 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:46.389456 (+ 2862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:46.389464 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:46.389483 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:46.389515 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:46.391689 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:46.392391 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:46.392397 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:46.392403 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:46.392549 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:46.392966 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:46.392971 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:46.392976 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:46.393576 (+ 600us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:46.393824 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:46.394811 (+ 987us) server_negotiation.cc:299] Negotiation successful
0619 12:28:46.395065 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":143,"threads_started":1}
I20260619 12:28:46.398891 28071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:46.399221 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:46.404798 28071 catalog_manager.cc:1347] Generated new cluster ID: 807c7e14e83f4021ba089da100ff62c9
I20260619 12:28:46.404868 28071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:46.412763 28060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:46.401333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:51131 (local address 127.0.0.1:42475)
0619 12:28:46.401504 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:46.401508 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:46.403672 (+ 2164us) server_negotiation.cc:407] Connection header received
0619 12:28:46.403826 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:46.403829 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:46.403892 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:46.403982 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:46.404867 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:46.405766 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:46.406981 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:46.407339 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:46.407444 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:46.409209 (+ 1765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:46.409213 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:46.409216 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:46.409240 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:46.410933 (+ 1693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:46.411489 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:46.411492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:46.411494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:46.411571 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:46.411907 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:46.411910 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:46.411912 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:46.412099 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:46.412241 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:46.412507 (+ 266us) server_negotiation.cc:299] Negotiation successful
0619 12:28:46.412597 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260619 12:28:46.416594 28071 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:46.417721 28071 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:46.423377 28071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8807cf71ce944268d0e7d587eaf47f6: Generated new TSK 0
I20260619 12:28:46.424293 28071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:46.432757 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 27989
W20260619 12:28:46.438071 27974 connection.cc:535] client connection to 127.0.0.1:42475 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42475): Connection reset by peer (error 104)
2026-06-19T12:28:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1479 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:46 dist-test-slave-kr9h krb5kdc[28088](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:46 dist-test-slave-kr9h krb5kdc[28088](info): set up 2 sockets
Jun 19 12:28:46 dist-test-slave-kr9h krb5kdc[28088](info): commencing operation
krb5kdc: starting...
W20260619 12:28:47.474203 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:47 dist-test-slave-kr9h krb5kdc[28088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872127, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:28:47.602561 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38351
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42161
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:47.718788 28101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:47.719084 28101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:47.719134 28101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:47.723646 28101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:47.723745 28101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:47.723765 28101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:47.723783 28101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:47.723799 28101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:47.729904 28101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42161
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38351
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:47.731081 28101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:47.732187 28101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:47.741827 28109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:47.741991 28101 server_base.cc:1034] running on GCE node
W20260619 12:28:47.741895 28106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:47.741878 28107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:47.742941 28101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:47.744184 28101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:47.745411 28101 hybrid_clock.cc:648] HybridClock initialized: now 1781872127745396 us; error 31 us; skew 500 ppm
Jun 19 12:28:47 dist-test-slave-kr9h krb5kdc[28088](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872127, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:47.748389 28101 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:28:47.749365 28101 webserver.cc:460] Webserver started at http://127.0.0.1:42085/ using document root <none> and password file <none>
I20260619 12:28:47.749972 28101 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:47.750036 28101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:47.750207 28101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:47.752316 28101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "cb2a61dc386d45329cc5cca39346ad63"
format_stamp: "Formatted at 2026-06-19 12:28:47 on dist-test-slave-kr9h"
server_key: "b82f7556ff25e16116cb44bfa384b6c8"
server_key_iv: "5778e94e2662528ecbc324b7e6a5a480"
server_key_version: "encryptionkey@0"
I20260619 12:28:47.752928 28101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "cb2a61dc386d45329cc5cca39346ad63"
format_stamp: "Formatted at 2026-06-19 12:28:47 on dist-test-slave-kr9h"
server_key: "b82f7556ff25e16116cb44bfa384b6c8"
server_key_iv: "5778e94e2662528ecbc324b7e6a5a480"
server_key_version: "encryptionkey@0"
I20260619 12:28:47.756892 28101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 12:28:47.759680 28116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:47.760990 28101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260619 12:28:47.761154 28101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "cb2a61dc386d45329cc5cca39346ad63"
format_stamp: "Formatted at 2026-06-19 12:28:47 on dist-test-slave-kr9h"
server_key: "b82f7556ff25e16116cb44bfa384b6c8"
server_key_iv: "5778e94e2662528ecbc324b7e6a5a480"
server_key_version: "encryptionkey@0"
I20260619 12:28:47.761338 28101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:47.809168 28101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:47.812793 28101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:47.813030 28101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:47.821031 28101 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38351
I20260619 12:28:47.821048 28168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38351 every 8 connection(s)
I20260619 12:28:47.822126 28101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:47.825070 28169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:47.830426 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 28101
I20260619 12:28:47.830577 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:47.830845 7071 external_mini_cluster.cc:1428] Setting key 92055f7cd50fcb4b3ce16e9589ae9ce2
I20260619 12:28:47.831578 28169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63: Bootstrap starting.
I20260619 12:28:47.834431 28169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:47.835423 28169 log.cc:824] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:47.837700 28169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63: No bootstrap required, opened a new log
I20260619 12:28:47.842029 28172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:47.832636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:57249 (local address 127.0.0.1:38351)
0619 12:28:47.833082 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:47.833104 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:47.833147 (+ 43us) server_negotiation.cc:407] Connection header received
0619 12:28:47.834037 (+ 890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:47.834060 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:47.834523 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:47.834939 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:47.835573 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:47.836965 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:47.838002 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:47.838682 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:47.838856 (+ 174us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:47.839777 (+ 921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:47.839783 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:47.839802 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:47.840332 (+ 530us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:47.840401 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:47.840431 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:47.840578 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:47.840725 (+ 147us) server_negotiation.cc:299] Negotiation successful
0619 12:28:47.840902 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":135,"threads_started":1}
I20260619 12:28:47.842630 28169 raft_consensus.cc:348] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2a61dc386d45329cc5cca39346ad63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38351 } }
I20260619 12:28:47.842890 28169 raft_consensus.cc:374] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:47.842975 28169 raft_consensus.cc:729] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb2a61dc386d45329cc5cca39346ad63, State: Initialized, Role: FOLLOWER
I20260619 12:28:47.843482 28169 consensus_queue.cc:260] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2a61dc386d45329cc5cca39346ad63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38351 } }
I20260619 12:28:47.843637 28169 raft_consensus.cc:388] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:47.843701 28169 raft_consensus.cc:482] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:47.843797 28169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:47.845072 28169 raft_consensus.cc:504] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2a61dc386d45329cc5cca39346ad63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38351 } }
I20260619 12:28:47.845274 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:47.845501 28169 leader_election.cc:302] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cb2a61dc386d45329cc5cca39346ad63; no voters:
I20260619 12:28:47.845853 28169 leader_election.cc:288] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:47.845988 28174 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:47.846238 28174 raft_consensus.cc:686] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [term 1 LEADER]: Becoming Leader. State: Replica: cb2a61dc386d45329cc5cca39346ad63, State: Running, Role: LEADER
I20260619 12:28:47.846606 28174 consensus_queue.cc:237] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2a61dc386d45329cc5cca39346ad63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38351 } }
I20260619 12:28:47.847247 28169 sys_catalog.cc:564] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:47.848408 28175 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cb2a61dc386d45329cc5cca39346ad63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2a61dc386d45329cc5cca39346ad63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38351 } } }
I20260619 12:28:47.848531 28175 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:47.848857 28176 sys_catalog.cc:455] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cb2a61dc386d45329cc5cca39346ad63. Latest consensus state: current_term: 1 leader_uuid: "cb2a61dc386d45329cc5cca39346ad63" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cb2a61dc386d45329cc5cca39346ad63" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38351 } } }
I20260619 12:28:47.848960 28176 sys_catalog.cc:458] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:47.849967 28188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:47.852655 28172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:47.847657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:56779 (local address 127.0.0.1:38351)
0619 12:28:47.847777 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:47.847781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:47.848167 (+ 386us) server_negotiation.cc:407] Connection header received
0619 12:28:47.848382 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:47.848386 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:47.848437 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:47.848509 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:47.848928 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:47.849721 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:47.850747 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:47.851000 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:47.851117 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:47.852141 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:47.852146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:47.852148 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:47.852163 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:47.852197 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:47.852209 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:47.852311 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:47.852457 (+ 146us) server_negotiation.cc:299] Negotiation successful
0619 12:28:47.852493 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 12:28:47.853570 28188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:47.859714 28188 catalog_manager.cc:1347] Generated new cluster ID: 8004f75216ce4640aceec790dcf70d4b
I20260619 12:28:47.859810 28188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:47.879710 28188 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:47.880913 28188 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:47.894326 28188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cb2a61dc386d45329cc5cca39346ad63: Generated new TSK 0
I20260619 12:28:47.895134 28188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:47.904177 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 28101
W20260619 12:28:47.909626 28086 connection.cc:535] client connection to 127.0.0.1:38351 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38351): Connection reset by peer (error 104)
2026-06-19T12:28:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1471 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:47 dist-test-slave-kr9h krb5kdc[28200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:47 dist-test-slave-kr9h krb5kdc[28200](info): set up 2 sockets
Jun 19 12:28:47 dist-test-slave-kr9h krb5kdc[28200](info): commencing operation
krb5kdc: starting...
W20260619 12:28:48.978363 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.054s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:49 dist-test-slave-kr9h krb5kdc[28200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:28:49.104524 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40237
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36989
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40237
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:49.217202 28213 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:49.217559 28213 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:49.217648 28213 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:49.221824 28213 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:49.221931 28213 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:49.221966 28213 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:49.222004 28213 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:49.222036 28213 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:49.227510 28213 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36989
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40237
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40237
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28213
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:49.228770 28213 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:49.229940 28213 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:28:49.240842 28213 server_base.cc:1034] running on GCE node
W20260619 12:28:49.240842 28219 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:49.240865 28218 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:49.241138 28221 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:49.241803 28213 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:49.242990 28213 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:49.244186 28213 hybrid_clock.cc:648] HybridClock initialized: now 1781872129244155 us; error 45 us; skew 500 ppm
Jun 19 12:28:49 dist-test-slave-kr9h krb5kdc[28200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872129, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:49.247136 28213 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:28:49.248174 28213 webserver.cc:460] Webserver started at http://127.0.0.1:36183/ using document root <none> and password file <none>
I20260619 12:28:49.248834 28213 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:49.248914 28213 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:49.249187 28213 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:49.251586 28213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "16bc4f508a6548338b10dec67d34fd90"
format_stamp: "Formatted at 2026-06-19 12:28:49 on dist-test-slave-kr9h"
server_key: "72eaa285986a197f1751cbb0997dcce7"
server_key_iv: "8935b55657f75ad6311b0ff970054acb"
server_key_version: "encryptionkey@0"
I20260619 12:28:49.252300 28213 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "16bc4f508a6548338b10dec67d34fd90"
format_stamp: "Formatted at 2026-06-19 12:28:49 on dist-test-slave-kr9h"
server_key: "72eaa285986a197f1751cbb0997dcce7"
server_key_iv: "8935b55657f75ad6311b0ff970054acb"
server_key_version: "encryptionkey@0"
I20260619 12:28:49.256264 28213 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:28:49.258917 28228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:49.260257 28213 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:28:49.260421 28213 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "16bc4f508a6548338b10dec67d34fd90"
format_stamp: "Formatted at 2026-06-19 12:28:49 on dist-test-slave-kr9h"
server_key: "72eaa285986a197f1751cbb0997dcce7"
server_key_iv: "8935b55657f75ad6311b0ff970054acb"
server_key_version: "encryptionkey@0"
I20260619 12:28:49.260614 28213 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:49.265539 28213 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:49.269025 28213 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:49.269241 28213 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:49.276575 28213 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40237
I20260619 12:28:49.276638 28280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40237 every 8 connection(s)
I20260619 12:28:49.277743 28213 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:49.280731 28281 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:49.281409 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 28213
I20260619 12:28:49.281579 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:49.281893 7071 external_mini_cluster.cc:1428] Setting key 58c088afb24033553d7be19ab357e6cd
I20260619 12:28:49.287360 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:49.283385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:51011 (local address 127.0.0.1:40237)
0619 12:28:49.283823 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:49.283838 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:49.283873 (+ 35us) server_negotiation.cc:407] Connection header received
0619 12:28:49.284531 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:49.284549 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:49.284840 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:49.285160 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:49.286028 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:49.286033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:49.286045 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:49.286114 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:49.286169 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:49.286190 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:49.286303 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:49.286415 (+ 112us) server_negotiation.cc:299] Negotiation successful
0619 12:28:49.286607 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279,"thread_start_us":146,"threads_started":1}
I20260619 12:28:49.287451 28281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90: Bootstrap starting.
I20260619 12:28:49.289803 28281 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:49.290247 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:49.290642 28281 log.cc:824] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:49.292819 28281 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90: No bootstrap required, opened a new log
I20260619 12:28:49.293859 28284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:49.292260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:48001 (local address 127.0.0.1:40237)
0619 12:28:49.292388 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:49.292393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:49.292526 (+ 133us) server_negotiation.cc:407] Connection header received
0619 12:28:49.292635 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:49.292638 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:49.292672 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:49.292770 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:49.293504 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:49.293507 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:49.293510 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:49.293521 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:49.293547 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:49.293560 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:49.293649 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:49.293703 (+ 54us) server_negotiation.cc:299] Negotiation successful
0619 12:28:49.293748 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:28:49.296761 28281 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16bc4f508a6548338b10dec67d34fd90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40237 } }
I20260619 12:28:49.296988 28281 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:49.297027 28281 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16bc4f508a6548338b10dec67d34fd90, State: Initialized, Role: FOLLOWER
I20260619 12:28:49.297505 28281 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16bc4f508a6548338b10dec67d34fd90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40237 } }
I20260619 12:28:49.297642 28281 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:49.297691 28281 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:49.297765 28281 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:49.298717 28281 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16bc4f508a6548338b10dec67d34fd90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40237 } }
I20260619 12:28:49.299062 28281 leader_election.cc:302] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16bc4f508a6548338b10dec67d34fd90; no voters:
I20260619 12:28:49.299446 28281 leader_election.cc:288] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:49.299587 28291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:49.299798 28291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [term 1 LEADER]: Becoming Leader. State: Replica: 16bc4f508a6548338b10dec67d34fd90, State: Running, Role: LEADER
I20260619 12:28:49.300125 28291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16bc4f508a6548338b10dec67d34fd90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40237 } }
I20260619 12:28:49.300782 28281 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:49.301993 28292 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16bc4f508a6548338b10dec67d34fd90. Latest consensus state: current_term: 1 leader_uuid: "16bc4f508a6548338b10dec67d34fd90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16bc4f508a6548338b10dec67d34fd90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40237 } } }
I20260619 12:28:49.302136 28292 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:49.302417 28293 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16bc4f508a6548338b10dec67d34fd90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16bc4f508a6548338b10dec67d34fd90" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40237 } } }
I20260619 12:28:49.302544 28300 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:49.302584 28293 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:49.306031 28300 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:49.312094 28300 catalog_manager.cc:1347] Generated new cluster ID: d405c71f203a4cbf9ed94e016d3fdac1
I20260619 12:28:49.312208 28300 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:49.331473 28300 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:49.331849 28300 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:49.343804 28300 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16bc4f508a6548338b10dec67d34fd90: Generated new TSK 0
I20260619 12:28:49.344743 28300 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:49.352701 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 28213
2026-06-19T12:28:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1448 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:49 dist-test-slave-kr9h krb5kdc[28312](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:49 dist-test-slave-kr9h krb5kdc[28312](info): set up 2 sockets
Jun 19 12:28:49 dist-test-slave-kr9h krb5kdc[28312](info): commencing operation
krb5kdc: starting...
W20260619 12:28:50.397864 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:50 dist-test-slave-kr9h krb5kdc[28312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:28:50.524971 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34883
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34097
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:50.643193 28325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:50.643501 28325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:50.643545 28325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:50.647804 28325 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:50.647886 28325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:50.647907 28325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:50.647927 28325 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:50.647943 28325 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:50.653673 28325 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34097
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34883
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28325
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:50.654873 28325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:50.655901 28325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:50.665316 28333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:50.665542 28330 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:50.665325 28331 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:50.665763 28325 server_base.cc:1034] running on GCE node
I20260619 12:28:50.666414 28325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:50.667709 28325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:50.668929 28325 hybrid_clock.cc:648] HybridClock initialized: now 1781872130668900 us; error 46 us; skew 500 ppm
Jun 19 12:28:50 dist-test-slave-kr9h krb5kdc[28312](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872130, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:50.672391 28325 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:28:50.673336 28325 webserver.cc:460] Webserver started at http://127.0.0.1:43663/ using document root <none> and password file <none>
I20260619 12:28:50.673974 28325 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:50.674042 28325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:50.674214 28325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:50.676338 28325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "640ef00a5e9f495580d38c57bc1e94f5"
format_stamp: "Formatted at 2026-06-19 12:28:50 on dist-test-slave-kr9h"
server_key: "b49c340f1222d87dbabe8b66f611b65f"
server_key_iv: "045595d4d817765029bd827f9e006fd8"
server_key_version: "encryptionkey@0"
I20260619 12:28:50.676913 28325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "640ef00a5e9f495580d38c57bc1e94f5"
format_stamp: "Formatted at 2026-06-19 12:28:50 on dist-test-slave-kr9h"
server_key: "b49c340f1222d87dbabe8b66f611b65f"
server_key_iv: "045595d4d817765029bd827f9e006fd8"
server_key_version: "encryptionkey@0"
I20260619 12:28:50.680967 28325 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260619 12:28:50.683982 28340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:50.685602 28325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:50.685738 28325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "640ef00a5e9f495580d38c57bc1e94f5"
format_stamp: "Formatted at 2026-06-19 12:28:50 on dist-test-slave-kr9h"
server_key: "b49c340f1222d87dbabe8b66f611b65f"
server_key_iv: "045595d4d817765029bd827f9e006fd8"
server_key_version: "encryptionkey@0"
I20260619 12:28:50.685916 28325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:50.715375 28325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:50.719244 28325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:50.719496 28325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:50.727433 28325 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34883
I20260619 12:28:50.727453 28392 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34883 every 8 connection(s)
I20260619 12:28:50.728536 28325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:50.731546 28393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:50.732847 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 28325
I20260619 12:28:50.733081 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:50.733415 7071 external_mini_cluster.cc:1428] Setting key 9eb61e253808f2579094a14cdc3b9c75
I20260619 12:28:50.738407 28393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5: Bootstrap starting.
I20260619 12:28:50.741204 28393 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:50.742234 28393 log.cc:824] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5: Log is configured to *not* fsync() on all Append() calls
Jun 19 12:28:50 dist-test-slave-kr9h krb5kdc[28312](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872130, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260619 12:28:50.744623 28393 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5: No bootstrap required, opened a new log
I20260619 12:28:50.748540 28393 raft_consensus.cc:348] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640ef00a5e9f495580d38c57bc1e94f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } }
I20260619 12:28:50.748801 28393 raft_consensus.cc:374] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:50.748883 28393 raft_consensus.cc:729] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 640ef00a5e9f495580d38c57bc1e94f5, State: Initialized, Role: FOLLOWER
I20260619 12:28:50.749297 28396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:50.734935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:34591 (local address 127.0.0.1:34883)
0619 12:28:50.735389 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:50.735406 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:50.735445 (+ 39us) server_negotiation.cc:407] Connection header received
0619 12:28:50.736130 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:50.736148 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:50.736519 (+ 371us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:50.736837 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:50.737814 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:50.738998 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:50.740249 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:50.740762 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:50.740894 (+ 132us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:50.744000 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:50.744006 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:50.744022 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:50.744050 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:50.746039 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:50.746623 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:50.746628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:50.746633 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:50.746745 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:50.747117 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:50.747120 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:50.747121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:50.747536 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:50.747786 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:50.748188 (+ 402us) server_negotiation.cc:299] Negotiation successful
0619 12:28:50.748407 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289,"thread_start_us":151,"threads_started":1}
I20260619 12:28:50.749434 28393 consensus_queue.cc:260] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640ef00a5e9f495580d38c57bc1e94f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } }
I20260619 12:28:50.749632 28393 raft_consensus.cc:388] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:50.749701 28393 raft_consensus.cc:482] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:50.749799 28393 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:50.750900 28393 raft_consensus.cc:504] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640ef00a5e9f495580d38c57bc1e94f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } }
I20260619 12:28:50.751279 28393 leader_election.cc:302] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 640ef00a5e9f495580d38c57bc1e94f5; no voters:
I20260619 12:28:50.751649 28393 leader_election.cc:288] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:50.751760 28398 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:50.751973 28398 raft_consensus.cc:686] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [term 1 LEADER]: Becoming Leader. State: Replica: 640ef00a5e9f495580d38c57bc1e94f5, State: Running, Role: LEADER
I20260619 12:28:50.752355 28398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640ef00a5e9f495580d38c57bc1e94f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } }
I20260619 12:28:50.752720 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:50.752977 28393 sys_catalog.cc:564] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:50.753588 28400 sys_catalog.cc:455] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 640ef00a5e9f495580d38c57bc1e94f5. Latest consensus state: current_term: 1 leader_uuid: "640ef00a5e9f495580d38c57bc1e94f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640ef00a5e9f495580d38c57bc1e94f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } } }
I20260619 12:28:50.753708 28400 sys_catalog.cc:458] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:50.754364 28399 sys_catalog.cc:455] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "640ef00a5e9f495580d38c57bc1e94f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "640ef00a5e9f495580d38c57bc1e94f5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34883 } } }
I20260619 12:28:50.754451 28399 sys_catalog.cc:458] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:50.754739 28404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:50.758510 28404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:50.765054 28396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:50.755708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:38167 (local address 127.0.0.1:34883)
0619 12:28:50.755817 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:50.755822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:50.756105 (+ 283us) server_negotiation.cc:407] Connection header received
0619 12:28:50.756241 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:50.756243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:50.756286 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:50.756352 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:50.757203 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:50.758067 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:50.759154 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:50.759542 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:50.759652 (+ 110us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:50.761205 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:50.761208 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:50.761211 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:50.761245 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:50.762867 (+ 1622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:50.763474 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:50.763478 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:50.763480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:50.763584 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:50.763976 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:50.763980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:50.763982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:50.764210 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:50.764356 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:50.764707 (+ 351us) server_negotiation.cc:299] Negotiation successful
0619 12:28:50.764814 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 12:28:50.764901 28404 catalog_manager.cc:1347] Generated new cluster ID: 09e25ab16a5943d8b8e3b9408e0997d3
I20260619 12:28:50.765236 28404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:50.771122 28404 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:50.772264 28404 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:50.783831 28404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 640ef00a5e9f495580d38c57bc1e94f5: Generated new TSK 0
I20260619 12:28:50.784816 28404 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:50.793831 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 28325
W20260619 12:28:50.798344 28310 connection.cc:535] client connection to 127.0.0.1:34883 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34883): Connection reset by peer (error 104)
2026-06-19T12:28:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:50 dist-test-slave-kr9h krb5kdc[28424](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:50 dist-test-slave-kr9h krb5kdc[28424](info): set up 2 sockets
Jun 19 12:28:50 dist-test-slave-kr9h krb5kdc[28424](info): commencing operation
krb5kdc: starting...
W20260619 12:28:51.833773 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.021s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:51 dist-test-slave-kr9h krb5kdc[28424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:28:51.958580 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:52.072520 28437 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:52.072811 28437 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:52.072854 28437 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:52.076939 28437 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:52.077018 28437 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:52.077039 28437 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:52.077055 28437 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:52.077071 28437 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:52.082562 28437 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39359
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28437
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:52.083686 28437 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:52.084743 28437 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:52.093302 28442 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:52.093314 28445 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:52.093358 28437 server_base.cc:1034] running on GCE node
W20260619 12:28:52.093314 28443 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:52.094250 28437 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:52.095434 28437 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:52.096616 28437 hybrid_clock.cc:648] HybridClock initialized: now 1781872132096597 us; error 35 us; skew 500 ppm
Jun 19 12:28:52 dist-test-slave-kr9h krb5kdc[28424](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872132, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:52.104465 28437 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:28:52.105706 28437 webserver.cc:460] Webserver started at http://127.0.0.1:38621/ using document root <none> and password file <none>
I20260619 12:28:52.106307 28437 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:52.106374 28437 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:52.106633 28437 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:52.108883 28437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "21b97ba82b8e462f8e1a5c6db34ef460"
format_stamp: "Formatted at 2026-06-19 12:28:52 on dist-test-slave-kr9h"
server_key: "9b66b1525f0261fcae85d684760e4638"
server_key_iv: "d8d945cb994bc204ed0e756785a79449"
server_key_version: "encryptionkey@0"
I20260619 12:28:52.109582 28437 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "21b97ba82b8e462f8e1a5c6db34ef460"
format_stamp: "Formatted at 2026-06-19 12:28:52 on dist-test-slave-kr9h"
server_key: "9b66b1525f0261fcae85d684760e4638"
server_key_iv: "d8d945cb994bc204ed0e756785a79449"
server_key_version: "encryptionkey@0"
I20260619 12:28:52.113597 28437 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 12:28:52.116308 28452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:52.117909 28437 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:52.118108 28437 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "21b97ba82b8e462f8e1a5c6db34ef460"
format_stamp: "Formatted at 2026-06-19 12:28:52 on dist-test-slave-kr9h"
server_key: "9b66b1525f0261fcae85d684760e4638"
server_key_iv: "d8d945cb994bc204ed0e756785a79449"
server_key_version: "encryptionkey@0"
I20260619 12:28:52.118220 28437 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:52.131326 28437 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:52.134517 28437 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:52.134734 28437 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:52.142014 28437 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39359
I20260619 12:28:52.142035 28504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39359 every 8 connection(s)
I20260619 12:28:52.143036 28437 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:52.145390 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 28437
I20260619 12:28:52.145587 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:52.145891 7071 external_mini_cluster.cc:1428] Setting key b14c9b7875284bd684affcae5c246c12
I20260619 12:28:52.145956 28505 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:52.153298 28505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460: Bootstrap starting.
I20260619 12:28:52.156106 28505 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:52.156338 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:52.147332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:33437 (local address 127.0.0.1:39359)
0619 12:28:52.147792 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:52.147811 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:52.147853 (+ 42us) server_negotiation.cc:407] Connection header received
0619 12:28:52.148758 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:52.148784 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:52.149238 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:52.149649 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:52.150076 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:52.151528 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:52.152429 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:52.153128 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:52.153290 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:52.154308 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:52.154315 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:52.154330 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:52.154876 (+ 546us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:52.154942 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:52.154971 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:52.155101 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:52.155268 (+ 167us) server_negotiation.cc:299] Negotiation successful
0619 12:28:52.155394 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":136,"threads_started":1}
I20260619 12:28:52.156935 28505 log.cc:824] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:52.159126 28505 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460: No bootstrap required, opened a new log
I20260619 12:28:52.159389 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:52.163919 28505 raft_consensus.cc:348] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b97ba82b8e462f8e1a5c6db34ef460" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39359 } }
I20260619 12:28:52.164213 28505 raft_consensus.cc:374] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:52.164264 28505 raft_consensus.cc:729] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21b97ba82b8e462f8e1a5c6db34ef460, State: Initialized, Role: FOLLOWER
I20260619 12:28:52.164744 28505 consensus_queue.cc:260] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b97ba82b8e462f8e1a5c6db34ef460" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39359 } }
I20260619 12:28:52.164937 28505 raft_consensus.cc:388] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:52.165007 28505 raft_consensus.cc:482] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:52.165112 28505 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:52.165835 28508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:52.161680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:43973 (local address 127.0.0.1:39359)
0619 12:28:52.161820 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:52.161826 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:52.162093 (+ 267us) server_negotiation.cc:407] Connection header received
0619 12:28:52.162186 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:52.162189 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:52.162234 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:52.162308 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:52.162543 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:52.163320 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:52.164122 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:52.164452 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:52.164557 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:52.165400 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:52.165404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:52.165406 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:52.165419 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:52.165468 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:52.165483 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:52.165578 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:52.165666 (+ 88us) server_negotiation.cc:299] Negotiation successful
0619 12:28:52.165699 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260619 12:28:52.166324 28505 raft_consensus.cc:504] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b97ba82b8e462f8e1a5c6db34ef460" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39359 } }
I20260619 12:28:52.166675 28505 leader_election.cc:302] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 21b97ba82b8e462f8e1a5c6db34ef460; no voters:
I20260619 12:28:52.166986 28505 leader_election.cc:288] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:52.167124 28515 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:52.167357 28515 raft_consensus.cc:686] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [term 1 LEADER]: Becoming Leader. State: Replica: 21b97ba82b8e462f8e1a5c6db34ef460, State: Running, Role: LEADER
I20260619 12:28:52.167699 28515 consensus_queue.cc:237] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b97ba82b8e462f8e1a5c6db34ef460" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39359 } }
I20260619 12:28:52.168249 28505 sys_catalog.cc:564] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:52.169415 28516 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "21b97ba82b8e462f8e1a5c6db34ef460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b97ba82b8e462f8e1a5c6db34ef460" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39359 } } }
I20260619 12:28:52.169456 28517 sys_catalog.cc:455] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 21b97ba82b8e462f8e1a5c6db34ef460. Latest consensus state: current_term: 1 leader_uuid: "21b97ba82b8e462f8e1a5c6db34ef460" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "21b97ba82b8e462f8e1a5c6db34ef460" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39359 } } }
I20260619 12:28:52.169559 28516 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:52.169560 28517 sys_catalog.cc:458] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:52.169987 28524 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:52.172894 28524 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:52.178758 28524 catalog_manager.cc:1347] Generated new cluster ID: 41f5166a49db405ab7adc5dc1a8b46c6
I20260619 12:28:52.178848 28524 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:52.213874 28524 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:52.214905 28524 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:52.221462 28524 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 21b97ba82b8e462f8e1a5c6db34ef460: Generated new TSK 0
I20260619 12:28:52.222285 28524 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:52.234875 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 28437
W20260619 12:28:52.240456 28422 connection.cc:535] client connection to 127.0.0.1:39359 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39359): Connection reset by peer (error 104)
2026-06-19T12:28:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1442 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:52 dist-test-slave-kr9h krb5kdc[28536](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:52 dist-test-slave-kr9h krb5kdc[28536](info): set up 2 sockets
Jun 19 12:28:52 dist-test-slave-kr9h krb5kdc[28536](info): commencing operation
krb5kdc: starting...
W20260619 12:28:53.285809 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:53 dist-test-slave-kr9h krb5kdc[28536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260619 12:28:53.409503 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40891
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36563
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40891
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:53.522979 28549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:53.523281 28549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:53.523325 28549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:53.527354 28549 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:53.527423 28549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:53.527443 28549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:53.527459 28549 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:53.527474 28549 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:53.532899 28549 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36563
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40891
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:53.534161 28549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:53.535229 28549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:53.544060 28557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:53.544092 28554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:53.544135 28549 server_base.cc:1034] running on GCE node
W20260619 12:28:53.544092 28555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:53.544893 28549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:53.546157 28549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:53.547365 28549 hybrid_clock.cc:648] HybridClock initialized: now 1781872133547329 us; error 51 us; skew 500 ppm
Jun 19 12:28:53 dist-test-slave-kr9h krb5kdc[28536](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872133, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:53.550618 28549 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260619 12:28:53.551738 28549 webserver.cc:460] Webserver started at http://127.0.0.1:39067/ using document root <none> and password file <none>
I20260619 12:28:53.552363 28549 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:53.552481 28549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:53.552735 28549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:53.555058 28549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "1700059b9f004c6fa04541655bb88fd2"
format_stamp: "Formatted at 2026-06-19 12:28:53 on dist-test-slave-kr9h"
server_key: "e90cb4ae34fc26024c60fc2b5a15e4e9"
server_key_iv: "8f89faf058583a9399d16de624252e72"
server_key_version: "encryptionkey@0"
I20260619 12:28:53.555685 28549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "1700059b9f004c6fa04541655bb88fd2"
format_stamp: "Formatted at 2026-06-19 12:28:53 on dist-test-slave-kr9h"
server_key: "e90cb4ae34fc26024c60fc2b5a15e4e9"
server_key_iv: "8f89faf058583a9399d16de624252e72"
server_key_version: "encryptionkey@0"
I20260619 12:28:53.559867 28549 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:28:53.562650 28564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:53.563987 28549 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:53.564141 28549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "1700059b9f004c6fa04541655bb88fd2"
format_stamp: "Formatted at 2026-06-19 12:28:53 on dist-test-slave-kr9h"
server_key: "e90cb4ae34fc26024c60fc2b5a15e4e9"
server_key_iv: "8f89faf058583a9399d16de624252e72"
server_key_version: "encryptionkey@0"
I20260619 12:28:53.564266 28549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:53.569288 28549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:53.572861 28549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:53.573102 28549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:53.580576 28549 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40891
I20260619 12:28:53.580595 28616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40891 every 8 connection(s)
I20260619 12:28:53.581697 28549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:53.584689 28617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:53.586045 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 28549
I20260619 12:28:53.586203 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:53.586503 7071 external_mini_cluster.cc:1428] Setting key c3269e841ed60c28664ad601703fcec3
I20260619 12:28:53.591715 28617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2: Bootstrap starting.
I20260619 12:28:53.592021 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:53.587956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:34593 (local address 127.0.0.1:40891)
0619 12:28:53.588419 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:53.588435 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:53.588473 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:28:53.589125 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:53.589144 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:53.589511 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:53.589806 (+ 295us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:53.590690 (+ 884us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:53.590694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:53.590706 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:53.590716 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:53.590768 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:53.590787 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:53.590901 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:53.591022 (+ 121us) server_negotiation.cc:299] Negotiation successful
0619 12:28:53.591238 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":153,"threads_started":1}
I20260619 12:28:53.594238 28617 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:53.595062 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:53.595045 28617 log.cc:824] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:53.597599 28617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2: No bootstrap required, opened a new log
I20260619 12:28:53.599206 28620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:53.597369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.28:52179 (local address 127.0.0.1:40891)
0619 12:28:53.597521 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:53.597526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:53.597799 (+ 273us) server_negotiation.cc:407] Connection header received
0619 12:28:53.597909 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:53.597912 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:53.597952 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:53.598051 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:53.598816 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:53.598820 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:53.598823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:53.598835 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:53.598865 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:53.598878 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:53.598968 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:53.599025 (+ 57us) server_negotiation.cc:299] Negotiation successful
0619 12:28:53.599068 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260619 12:28:53.601891 28617 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1700059b9f004c6fa04541655bb88fd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } }
I20260619 12:28:53.602146 28617 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:53.602181 28617 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1700059b9f004c6fa04541655bb88fd2, State: Initialized, Role: FOLLOWER
I20260619 12:28:53.602566 28617 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1700059b9f004c6fa04541655bb88fd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } }
I20260619 12:28:53.602694 28617 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:53.602741 28617 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:53.602813 28617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:53.603781 28617 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1700059b9f004c6fa04541655bb88fd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } }
I20260619 12:28:53.604208 28617 leader_election.cc:302] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1700059b9f004c6fa04541655bb88fd2; no voters:
I20260619 12:28:53.604594 28617 leader_election.cc:288] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:53.604727 28627 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:53.604990 28627 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [term 1 LEADER]: Becoming Leader. State: Replica: 1700059b9f004c6fa04541655bb88fd2, State: Running, Role: LEADER
I20260619 12:28:53.605471 28627 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1700059b9f004c6fa04541655bb88fd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } }
I20260619 12:28:53.606052 28617 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:53.607213 28628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1700059b9f004c6fa04541655bb88fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1700059b9f004c6fa04541655bb88fd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } } }
I20260619 12:28:53.607352 28628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:53.607702 28629 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1700059b9f004c6fa04541655bb88fd2. Latest consensus state: current_term: 1 leader_uuid: "1700059b9f004c6fa04541655bb88fd2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1700059b9f004c6fa04541655bb88fd2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40891 } } }
I20260619 12:28:53.607780 28636 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:53.607858 28629 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:53.610711 28636 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:53.616294 28636 catalog_manager.cc:1347] Generated new cluster ID: 419a57dcb19848809fc120ce8dd10b29
I20260619 12:28:53.616370 28636 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:53.626400 28636 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:53.626739 28636 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:53.633419 28636 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1700059b9f004c6fa04541655bb88fd2: Generated new TSK 0
I20260619 12:28:53.634155 28636 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:53.645707 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 28549
2026-06-19T12:28:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1410 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:53 dist-test-slave-kr9h krb5kdc[28648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:53 dist-test-slave-kr9h krb5kdc[28648](info): set up 2 sockets
Jun 19 12:28:53 dist-test-slave-kr9h krb5kdc[28648](info): commencing operation
krb5kdc: starting...
W20260619 12:28:54.705888 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:54 dist-test-slave-kr9h krb5kdc[28648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:28:54.835088 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:43347
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:45765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:43347
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:54.946702 28661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:54.947026 28661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:54.947106 28661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:54.951201 28661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:54.951308 28661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:54.951356 28661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:54.951395 28661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:54.951429 28661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:54.957204 28661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:45765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:43347
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:43347
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28661
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:54.958384 28661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:54.959497 28661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:54.968837 28666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:54.968840 28667 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:54.968966 28669 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:54.969192 28661 server_base.cc:1034] running on GCE node
I20260619 12:28:54.969919 28661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:54.971064 28661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:54.972283 28661 hybrid_clock.cc:648] HybridClock initialized: now 1781872134972265 us; error 33 us; skew 500 ppm
Jun 19 12:28:54 dist-test-slave-kr9h krb5kdc[28648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872134, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:54.975332 28661 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:28:54.976255 28661 webserver.cc:460] Webserver started at http://127.6.231.254:46383/ using document root <none> and password file <none>
I20260619 12:28:54.976778 28661 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:54.976861 28661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:54.977046 28661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:54.979135 28661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "632b47f350294bfe9ab0a1f9e3209b32"
format_stamp: "Formatted at 2026-06-19 12:28:54 on dist-test-slave-kr9h"
server_key: "6ee72a8bbeafa43166acfaabd66f2c27"
server_key_iv: "68118121295cdcd0011c3a8cf50b997d"
server_key_version: "encryptionkey@0"
I20260619 12:28:54.979655 28661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "632b47f350294bfe9ab0a1f9e3209b32"
format_stamp: "Formatted at 2026-06-19 12:28:54 on dist-test-slave-kr9h"
server_key: "6ee72a8bbeafa43166acfaabd66f2c27"
server_key_iv: "68118121295cdcd0011c3a8cf50b997d"
server_key_version: "encryptionkey@0"
I20260619 12:28:54.983397 28661 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:28:54.985924 28676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:54.987210 28661 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:28:54.987311 28661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "632b47f350294bfe9ab0a1f9e3209b32"
format_stamp: "Formatted at 2026-06-19 12:28:54 on dist-test-slave-kr9h"
server_key: "6ee72a8bbeafa43166acfaabd66f2c27"
server_key_iv: "68118121295cdcd0011c3a8cf50b997d"
server_key_version: "encryptionkey@0"
I20260619 12:28:54.987392 28661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:55.006091 28661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:55.009562 28661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:55.009727 28661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:55.017107 28661 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:43347
I20260619 12:28:55.017141 28728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:43347 every 8 connection(s)
I20260619 12:28:55.018123 28661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:55.020882 28729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:55.021759 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 28661
I20260619 12:28:55.021901 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:55.022197 7071 external_mini_cluster.cc:1428] Setting key 44cd00a194858e1b4c86d081fc45060d
I20260619 12:28:55.027087 28729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32: Bootstrap starting.
I20260619 12:28:55.029321 28729 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:55.030104 28729 log.cc:824] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32: Log is configured to *not* fsync() on all Append() calls
Jun 19 12:28:55 dist-test-slave-kr9h krb5kdc[28648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872134, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:28:55.032449 28729 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32: No bootstrap required, opened a new log
I20260619 12:28:55.035952 28729 raft_consensus.cc:348] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632b47f350294bfe9ab0a1f9e3209b32" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43347 } }
I20260619 12:28:55.036212 28729 raft_consensus.cc:374] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:55.036290 28729 raft_consensus.cc:729] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 632b47f350294bfe9ab0a1f9e3209b32, State: Initialized, Role: FOLLOWER
I20260619 12:28:55.036574 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:55.023628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47779 (local address 127.6.231.254:43347)
0619 12:28:55.024094 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:55.024109 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:55.024145 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:28:55.024775 (+ 630us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:55.024792 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:55.025200 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:55.025595 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:55.026369 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:55.027398 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:55.028244 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:55.028711 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:55.028883 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:55.031683 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:55.031689 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:55.031702 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:55.031728 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:55.033571 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:55.034101 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:55.034111 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:55.034116 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:55.034226 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:55.034567 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:55.034572 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:55.034579 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:55.034956 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:55.035194 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:55.035489 (+ 295us) server_negotiation.cc:299] Negotiation successful
0619 12:28:55.035701 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315,"thread_start_us":130,"threads_started":1}
I20260619 12:28:55.036811 28729 consensus_queue.cc:260] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632b47f350294bfe9ab0a1f9e3209b32" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43347 } }
I20260619 12:28:55.036968 28729 raft_consensus.cc:388] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:55.037030 28729 raft_consensus.cc:482] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:55.037125 28729 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:55.038169 28729 raft_consensus.cc:504] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632b47f350294bfe9ab0a1f9e3209b32" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43347 } }
I20260619 12:28:55.038530 28729 leader_election.cc:302] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 632b47f350294bfe9ab0a1f9e3209b32; no voters:
I20260619 12:28:55.038884 28729 leader_election.cc:288] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:55.038997 28734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:55.039264 28734 raft_consensus.cc:686] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [term 1 LEADER]: Becoming Leader. State: Replica: 632b47f350294bfe9ab0a1f9e3209b32, State: Running, Role: LEADER
I20260619 12:28:55.039569 28734 consensus_queue.cc:237] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632b47f350294bfe9ab0a1f9e3209b32" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43347 } }
I20260619 12:28:55.039795 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:55.040144 28729 sys_catalog.cc:564] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:55.042245 28736 sys_catalog.cc:455] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "632b47f350294bfe9ab0a1f9e3209b32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632b47f350294bfe9ab0a1f9e3209b32" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43347 } } }
I20260619 12:28:55.042375 28735 sys_catalog.cc:455] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 632b47f350294bfe9ab0a1f9e3209b32. Latest consensus state: current_term: 1 leader_uuid: "632b47f350294bfe9ab0a1f9e3209b32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "632b47f350294bfe9ab0a1f9e3209b32" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 43347 } } }
I20260619 12:28:55.042932 28735 sys_catalog.cc:458] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:55.043123 28736 sys_catalog.cc:458] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32 [sys.catalog]: This master's current role is: LEADER
W20260619 12:28:55.045194 28752 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:28:55.045291 28752 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:28:55.045372 28754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:55.046288 28754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:55.053081 28732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:55.042735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34791 (local address 127.6.231.254:43347)
0619 12:28:55.042863 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:55.042867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:55.043726 (+ 859us) server_negotiation.cc:407] Connection header received
0619 12:28:55.043855 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:55.043858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:55.043908 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:55.043994 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:55.044810 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:55.045578 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:55.046720 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:55.047098 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:55.047218 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:55.049043 (+ 1825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:55.049047 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:55.049050 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:55.049076 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:55.050893 (+ 1817us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:55.051421 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:55.051424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:55.051426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:55.051549 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:55.051940 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:55.051944 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:55.051954 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:55.052171 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:55.052314 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:55.052625 (+ 311us) server_negotiation.cc:299] Negotiation successful
0619 12:28:55.052727 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260619 12:28:55.053517 28754 catalog_manager.cc:1347] Generated new cluster ID: 7249eb0f41e14989af49f3d20a833cb5
I20260619 12:28:55.053581 28754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:55.072549 28754 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:55.073552 28754 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:55.079164 28754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 632b47f350294bfe9ab0a1f9e3209b32: Generated new TSK 0
I20260619 12:28:55.079870 28754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:55.085709 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 28661
W20260619 12:28:55.091104 28646 connection.cc:535] client connection to 127.6.231.254:43347 recv error: Network error: failed to read from TLS socket (remote: 127.6.231.254:43347): Connection reset by peer (error 104)
2026-06-19T12:28:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1439 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:55 dist-test-slave-kr9h krb5kdc[28760](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:55 dist-test-slave-kr9h krb5kdc[28760](info): set up 2 sockets
Jun 19 12:28:55 dist-test-slave-kr9h krb5kdc[28760](info): commencing operation
krb5kdc: starting...
W20260619 12:28:56.130134 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:56 dist-test-slave-kr9h krb5kdc[28760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872136, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:28:56.256570 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:37235
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:36489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:37235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:56.370483 28773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:56.370779 28773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:56.370823 28773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:56.375063 28773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:56.375135 28773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:56.375154 28773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:56.375171 28773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:56.375186 28773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:56.380667 28773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:36489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:37235
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:37235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:56.381933 28773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:56.383041 28773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:56.392297 28779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:56.392319 28778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:56.392302 28781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:56.392668 28773 server_base.cc:1034] running on GCE node
I20260619 12:28:56.393240 28773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:56.394484 28773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:56.395666 28773 hybrid_clock.cc:648] HybridClock initialized: now 1781872136395647 us; error 33 us; skew 500 ppm
Jun 19 12:28:56 dist-test-slave-kr9h krb5kdc[28760](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872136, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:56.398545 28773 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:28:56.399495 28773 webserver.cc:460] Webserver started at http://127.6.231.254:45665/ using document root <none> and password file <none>
I20260619 12:28:56.400084 28773 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:56.400180 28773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:56.400416 28773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:56.402593 28773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "a46f20a7747e4734b1581344b51ab427"
format_stamp: "Formatted at 2026-06-19 12:28:56 on dist-test-slave-kr9h"
server_key: "5d1f0e835b303b37cd5eba1b5388d7cc"
server_key_iv: "020e8cb93ab87fe10ee5bb200b78fcca"
server_key_version: "encryptionkey@0"
I20260619 12:28:56.403192 28773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "a46f20a7747e4734b1581344b51ab427"
format_stamp: "Formatted at 2026-06-19 12:28:56 on dist-test-slave-kr9h"
server_key: "5d1f0e835b303b37cd5eba1b5388d7cc"
server_key_iv: "020e8cb93ab87fe10ee5bb200b78fcca"
server_key_version: "encryptionkey@0"
I20260619 12:28:56.407246 28773 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260619 12:28:56.409870 28788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:56.411266 28773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:56.411445 28773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "a46f20a7747e4734b1581344b51ab427"
format_stamp: "Formatted at 2026-06-19 12:28:56 on dist-test-slave-kr9h"
server_key: "5d1f0e835b303b37cd5eba1b5388d7cc"
server_key_iv: "020e8cb93ab87fe10ee5bb200b78fcca"
server_key_version: "encryptionkey@0"
I20260619 12:28:56.411579 28773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:56.424588 28773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:56.428298 28773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:56.428553 28773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:56.436220 28773 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:37235
I20260619 12:28:56.436299 28840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:37235 every 8 connection(s)
I20260619 12:28:56.437274 28773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:56.440261 28841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:56.443559 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 28773
I20260619 12:28:56.443732 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:56.444065 7071 external_mini_cluster.cc:1428] Setting key 773524a9711a111de774903179a2fde6
I20260619 12:28:56.446596 28841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427: Bootstrap starting.
I20260619 12:28:56.449386 28841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:56.450309 28841 log.cc:824] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:56.452679 28841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427: No bootstrap required, opened a new log
I20260619 12:28:56.454596 28844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:56.445627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42785 (local address 127.6.231.254:37235)
0619 12:28:56.446087 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:56.446106 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:56.446149 (+ 43us) server_negotiation.cc:407] Connection header received
0619 12:28:56.446983 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:56.447006 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:56.447434 (+ 428us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:56.447811 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:56.448256 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:56.449778 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:56.450881 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:56.451491 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:56.451643 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:56.452524 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:56.452530 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:56.452545 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:56.453063 (+ 518us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:56.453127 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:56.453156 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:56.453277 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:56.453523 (+ 246us) server_negotiation.cc:299] Negotiation successful
0619 12:28:56.453662 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":126,"threads_started":1}
I20260619 12:28:56.457090 28841 raft_consensus.cc:348] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46f20a7747e4734b1581344b51ab427" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37235 } }
I20260619 12:28:56.457350 28841 raft_consensus.cc:374] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:56.457445 28841 raft_consensus.cc:729] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a46f20a7747e4734b1581344b51ab427, State: Initialized, Role: FOLLOWER
I20260619 12:28:56.457772 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:56.457973 28841 consensus_queue.cc:260] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46f20a7747e4734b1581344b51ab427" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37235 } }
I20260619 12:28:56.458114 28841 raft_consensus.cc:388] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:56.458178 28841 raft_consensus.cc:482] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:56.458280 28841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:56.459280 28841 raft_consensus.cc:504] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46f20a7747e4734b1581344b51ab427" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37235 } }
I20260619 12:28:56.459616 28841 leader_election.cc:302] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a46f20a7747e4734b1581344b51ab427; no voters:
I20260619 12:28:56.460068 28850 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:56.460352 28850 raft_consensus.cc:686] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [term 1 LEADER]: Becoming Leader. State: Replica: a46f20a7747e4734b1581344b51ab427, State: Running, Role: LEADER
I20260619 12:28:56.460094 28841 leader_election.cc:288] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:56.460680 28850 consensus_queue.cc:237] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46f20a7747e4734b1581344b51ab427" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37235 } }
I20260619 12:28:56.461870 28841 sys_catalog.cc:564] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:56.462512 28852 sys_catalog.cc:455] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a46f20a7747e4734b1581344b51ab427. Latest consensus state: current_term: 1 leader_uuid: "a46f20a7747e4734b1581344b51ab427" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46f20a7747e4734b1581344b51ab427" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37235 } } }
I20260619 12:28:56.462664 28852 sys_catalog.cc:458] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:56.462510 28853 sys_catalog.cc:455] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a46f20a7747e4734b1581344b51ab427" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a46f20a7747e4734b1581344b51ab427" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37235 } } }
I20260619 12:28:56.462757 28853 sys_catalog.cc:458] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:56.465045 28844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:56.459987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50635 (local address 127.6.231.254:37235)
0619 12:28:56.460098 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:56.460103 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:56.460381 (+ 278us) server_negotiation.cc:407] Connection header received
0619 12:28:56.460539 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:56.460543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:56.460593 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:56.460659 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:56.461073 (+ 414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:56.461827 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:56.463041 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:56.463356 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:56.463473 (+ 117us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:56.464495 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:56.464499 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:56.464501 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:56.464514 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:56.464540 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:56.464551 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:56.464616 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:56.464884 (+ 268us) server_negotiation.cc:299] Negotiation successful
0619 12:28:56.464928 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 12:28:56.467257 28866 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:28:56.467339 28866 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:28:56.467396 28859 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:56.468231 28859 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:56.474408 28859 catalog_manager.cc:1347] Generated new cluster ID: 4168e928e887441f99d952def65c10f1
I20260619 12:28:56.474489 28859 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:56.496271 28859 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:56.497697 28859 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:56.508277 28859 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a46f20a7747e4734b1581344b51ab427: Generated new TSK 0
I20260619 12:28:56.509111 28859 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:56.518653 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 28773
W20260619 12:28:56.524077 28758 connection.cc:535] client connection to 127.6.231.254:37235 recv error: Network error: failed to read from TLS socket (remote: 127.6.231.254:37235): Connection reset by peer (error 104)
2026-06-19T12:28:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1433 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:56 dist-test-slave-kr9h krb5kdc[28872](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:56 dist-test-slave-kr9h krb5kdc[28872](info): set up 2 sockets
Jun 19 12:28:56 dist-test-slave-kr9h krb5kdc[28872](info): commencing operation
krb5kdc: starting...
W20260619 12:28:57.570152 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.031s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:57 dist-test-slave-kr9h krb5kdc[28872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:28:57.697039 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:33291
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:38979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:33291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:57.809640 28885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:57.809994 28885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:57.810077 28885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:57.814218 28885 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:57.814321 28885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:57.814358 28885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:57.814393 28885 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:57.814435 28885 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:57.819828 28885 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:38979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:33291
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:33291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:57.820999 28885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:57.822170 28885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:57.831204 28891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:57.831207 28893 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:57.831207 28890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:57.831668 28885 server_base.cc:1034] running on GCE node
I20260619 12:28:57.832293 28885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:57.833556 28885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:57.834733 28885 hybrid_clock.cc:648] HybridClock initialized: now 1781872137834714 us; error 34 us; skew 500 ppm
Jun 19 12:28:57 dist-test-slave-kr9h krb5kdc[28872](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872137, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:57.846210 28885 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:28:57.847329 28885 webserver.cc:460] Webserver started at http://127.6.231.254:39493/ using document root <none> and password file <none>
I20260619 12:28:57.847945 28885 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:57.848011 28885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:57.848230 28885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:57.850523 28885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "8aa00d49d30040a0ac40ed430d29dfe3"
format_stamp: "Formatted at 2026-06-19 12:28:57 on dist-test-slave-kr9h"
server_key: "0f523ebad0e7716a34af343644e068cb"
server_key_iv: "83b16a23652a54cedb5ae06c007cb730"
server_key_version: "encryptionkey@0"
I20260619 12:28:57.851132 28885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "8aa00d49d30040a0ac40ed430d29dfe3"
format_stamp: "Formatted at 2026-06-19 12:28:57 on dist-test-slave-kr9h"
server_key: "0f523ebad0e7716a34af343644e068cb"
server_key_iv: "83b16a23652a54cedb5ae06c007cb730"
server_key_version: "encryptionkey@0"
I20260619 12:28:57.855072 28885 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:28:57.857733 28900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:57.859107 28885 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:28:57.859263 28885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "8aa00d49d30040a0ac40ed430d29dfe3"
format_stamp: "Formatted at 2026-06-19 12:28:57 on dist-test-slave-kr9h"
server_key: "0f523ebad0e7716a34af343644e068cb"
server_key_iv: "83b16a23652a54cedb5ae06c007cb730"
server_key_version: "encryptionkey@0"
I20260619 12:28:57.859390 28885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:57.864238 28885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:57.867491 28885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:57.867691 28885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:57.875133 28885 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:33291
I20260619 12:28:57.875160 28952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:33291 every 8 connection(s)
I20260619 12:28:57.876156 28885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:57.879423 28953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:57.883879 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 28885
I20260619 12:28:57.884023 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:57.884308 7071 external_mini_cluster.cc:1428] Setting key 25781490facd5b401e851e1c6eca42e1
I20260619 12:28:57.885792 28953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3: Bootstrap starting.
I20260619 12:28:57.888288 28953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:57.889150 28953 log.cc:824] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:57.890399 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:57.885857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49189 (local address 127.6.231.254:33291)
0619 12:28:57.886380 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:57.886397 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:57.886435 (+ 38us) server_negotiation.cc:407] Connection header received
0619 12:28:57.887140 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:57.887160 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:57.887476 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:57.887860 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:57.888784 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:57.888788 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:57.888802 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:57.888812 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:57.888867 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:57.888887 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:57.889009 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:57.889114 (+ 105us) server_negotiation.cc:299] Negotiation successful
0619 12:28:57.889317 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":135,"threads_started":1}
I20260619 12:28:57.891384 28953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3: No bootstrap required, opened a new log
I20260619 12:28:57.893565 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:57.895604 28953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa00d49d30040a0ac40ed430d29dfe3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33291 } }
I20260619 12:28:57.895960 28953 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:57.896044 28953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8aa00d49d30040a0ac40ed430d29dfe3, State: Initialized, Role: FOLLOWER
I20260619 12:28:57.896476 28953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa00d49d30040a0ac40ed430d29dfe3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33291 } }
I20260619 12:28:57.896627 28953 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:57.896718 28953 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:57.896826 28953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:57.897583 28956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:57.895732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58609 (local address 127.6.231.254:33291)
0619 12:28:57.895892 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:57.895897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:57.896209 (+ 312us) server_negotiation.cc:407] Connection header received
0619 12:28:57.896274 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:57.896278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:57.896319 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:57.896397 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:57.897165 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:57.897168 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:57.897171 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:28:57.897183 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:57.897223 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:28:57.897235 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:57.897330 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:57.897402 (+ 72us) server_negotiation.cc:299] Negotiation successful
0619 12:28:57.897463 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260619 12:28:57.898108 28953 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa00d49d30040a0ac40ed430d29dfe3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33291 } }
I20260619 12:28:57.898475 28953 leader_election.cc:302] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8aa00d49d30040a0ac40ed430d29dfe3; no voters:
I20260619 12:28:57.898928 28953 leader_election.cc:288] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:57.898994 28963 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:57.899257 28963 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [term 1 LEADER]: Becoming Leader. State: Replica: 8aa00d49d30040a0ac40ed430d29dfe3, State: Running, Role: LEADER
I20260619 12:28:57.899606 28963 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa00d49d30040a0ac40ed430d29dfe3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33291 } }
I20260619 12:28:57.900306 28953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:57.901348 28965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8aa00d49d30040a0ac40ed430d29dfe3. Latest consensus state: current_term: 1 leader_uuid: "8aa00d49d30040a0ac40ed430d29dfe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa00d49d30040a0ac40ed430d29dfe3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33291 } } }
I20260619 12:28:57.901505 28965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:57.901886 28972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:57.901846 28964 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8aa00d49d30040a0ac40ed430d29dfe3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8aa00d49d30040a0ac40ed430d29dfe3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 33291 } } }
I20260619 12:28:57.902029 28964 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3 [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:57.904592 28972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:57.910722 28972 catalog_manager.cc:1347] Generated new cluster ID: a4e11a719b43480b8a6f2f708242f6ab
I20260619 12:28:57.910812 28972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:57.928426 28972 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:57.928792 28972 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:57.941601 28972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8aa00d49d30040a0ac40ed430d29dfe3: Generated new TSK 0
I20260619 12:28:57.942368 28972 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:57.951189 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 28885
2026-06-19T12:28:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1432 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:58 dist-test-slave-kr9h krb5kdc[28984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:58 dist-test-slave-kr9h krb5kdc[28984](info): set up 2 sockets
Jun 19 12:28:58 dist-test-slave-kr9h krb5kdc[28984](info): commencing operation
krb5kdc: starting...
W20260619 12:28:59.010013 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:28:59 dist-test-slave-kr9h krb5kdc[28984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:28:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:28:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:28:59.136579 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:41209
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:33795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:41209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:28:59.251396 28997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:28:59.251760 28997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:28:59.251866 28997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:28:59.256122 28997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:28:59.256232 28997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:28:59.256273 28997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:28:59.256311 28997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:28:59.256350 28997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:28:59.262180 28997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:33795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:41209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:41209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:28:59.263399 28997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:28:59.264508 28997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:28:59.273424 29005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:59.273552 28997 server_base.cc:1034] running on GCE node
W20260619 12:28:59.273465 29003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:28:59.273736 29002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:28:59.274292 28997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:28:59.275415 28997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:28:59.276621 28997 hybrid_clock.cc:648] HybridClock initialized: now 1781872139276606 us; error 28 us; skew 500 ppm
Jun 19 12:28:59 dist-test-slave-kr9h krb5kdc[28984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872139, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:28:59.279475 28997 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:28:59.280325 28997 webserver.cc:460] Webserver started at http://127.6.231.254:41005/ using document root <none> and password file <none>
I20260619 12:28:59.280857 28997 fs_manager.cc:359] Metadata directory not provided
I20260619 12:28:59.280915 28997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:28:59.281078 28997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:28:59.283213 28997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "64dc09498a18477fb88987f736b8c3bd"
format_stamp: "Formatted at 2026-06-19 12:28:59 on dist-test-slave-kr9h"
server_key: "e4063f9140d58546f9a311c8a2dc11dd"
server_key_iv: "ba4e20543e8cdcc0b64b973d8b2fd6bd"
server_key_version: "encryptionkey@0"
I20260619 12:28:59.283706 28997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "64dc09498a18477fb88987f736b8c3bd"
format_stamp: "Formatted at 2026-06-19 12:28:59 on dist-test-slave-kr9h"
server_key: "e4063f9140d58546f9a311c8a2dc11dd"
server_key_iv: "ba4e20543e8cdcc0b64b973d8b2fd6bd"
server_key_version: "encryptionkey@0"
I20260619 12:28:59.287516 28997 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260619 12:28:59.290050 29012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:28:59.291374 28997 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:28:59.291531 28997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "64dc09498a18477fb88987f736b8c3bd"
format_stamp: "Formatted at 2026-06-19 12:28:59 on dist-test-slave-kr9h"
server_key: "e4063f9140d58546f9a311c8a2dc11dd"
server_key_iv: "ba4e20543e8cdcc0b64b973d8b2fd6bd"
server_key_version: "encryptionkey@0"
I20260619 12:28:59.291653 28997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:28:59.310976 28997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:28:59.314532 28997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:28:59.314766 28997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:28:59.321784 29064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:41209 every 8 connection(s)
I20260619 12:28:59.321871 28997 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:41209
I20260619 12:28:59.322777 28997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:28:59.323298 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 28997
I20260619 12:28:59.323446 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:28:59.323712 7071 external_mini_cluster.cc:1428] Setting key ce2c15bb6affaf6cd3893be288f63bf7
I20260619 12:28:59.325865 29065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:28:59.332415 29065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd: Bootstrap starting.
Jun 19 12:28:59 dist-test-slave-kr9h krb5kdc[28984](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872139, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:28:59.334666 29065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd: Neither blocks nor log segments found. Creating new log.
I20260619 12:28:59.335373 29065 log.cc:824] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd: Log is configured to *not* fsync() on all Append() calls
I20260619 12:28:59.337258 29065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd: No bootstrap required, opened a new log
I20260619 12:28:59.338198 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:59.325141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46343 (local address 127.6.231.254:41209)
0619 12:28:59.325558 (+ 417us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:59.325583 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:59.325619 (+ 36us) server_negotiation.cc:407] Connection header received
0619 12:28:59.326317 (+ 698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:59.326334 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:59.326670 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:59.326942 (+ 272us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:59.327820 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:59.328818 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:59.329970 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:59.330429 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:59.330542 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:59.333489 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:59.333498 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:59.333514 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:59.333549 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:59.335336 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:59.335805 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:59.335810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:59.335814 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:59.335923 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:59.336202 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:59.336206 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:59.336208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:59.336596 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:59.336793 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:59.337096 (+ 303us) server_negotiation.cc:299] Negotiation successful
0619 12:28:59.337480 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":262,"thread_start_us":142,"threads_started":1}
I20260619 12:28:59.340325 29065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64dc09498a18477fb88987f736b8c3bd" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41209 } }
I20260619 12:28:59.340559 29065 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:28:59.340634 29065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64dc09498a18477fb88987f736b8c3bd, State: Initialized, Role: FOLLOWER
I20260619 12:28:59.341094 29065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64dc09498a18477fb88987f736b8c3bd" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41209 } }
I20260619 12:28:59.341265 29065 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:28:59.341341 29065 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:28:59.341408 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:28:59.341486 29065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:28:59.342422 29065 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64dc09498a18477fb88987f736b8c3bd" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41209 } }
I20260619 12:28:59.342756 29065 leader_election.cc:302] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64dc09498a18477fb88987f736b8c3bd; no voters:
I20260619 12:28:59.343072 29065 leader_election.cc:288] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:28:59.343175 29074 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:28:59.343364 29074 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [term 1 LEADER]: Becoming Leader. State: Replica: 64dc09498a18477fb88987f736b8c3bd, State: Running, Role: LEADER
I20260619 12:28:59.343727 29074 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64dc09498a18477fb88987f736b8c3bd" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41209 } }
I20260619 12:28:59.344347 29065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:28:59.344887 29074 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64dc09498a18477fb88987f736b8c3bd. Latest consensus state: current_term: 1 leader_uuid: "64dc09498a18477fb88987f736b8c3bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64dc09498a18477fb88987f736b8c3bd" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41209 } } }
I20260619 12:28:59.345701 29074 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:59.345573 29075 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64dc09498a18477fb88987f736b8c3bd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64dc09498a18477fb88987f736b8c3bd" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 41209 } } }
I20260619 12:28:59.346028 29075 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd [sys.catalog]: This master's current role is: LEADER
I20260619 12:28:59.346874 29084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:28:59.349720 29084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:28:59.355485 29084 catalog_manager.cc:1347] Generated new cluster ID: c4c8a509cee7406c9c267e6451508e70
I20260619 12:28:59.355610 29084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:28:59.363384 29068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:28:59.343396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34263 (local address 127.6.231.254:41209)
0619 12:28:59.343502 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0619 12:28:59.343507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:28:59.343932 (+ 425us) server_negotiation.cc:407] Connection header received
0619 12:28:59.344109 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:28:59.344112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:28:59.344163 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:28:59.344244 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:28:59.345059 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:59.345870 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:59.346983 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:28:59.347354 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:28:59.347428 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:28:59.352296 (+ 4868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:28:59.352302 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:28:59.352311 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:28:59.352335 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:28:59.354319 (+ 1984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:59.354757 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:59.354762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:59.354764 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:59.354844 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:28:59.355128 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:28:59.355131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:28:59.355133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:28:59.355318 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:28:59.355453 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 12:28:59.363061 (+ 7608us) server_negotiation.cc:299] Negotiation successful
0619 12:28:59.363194 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260619 12:28:59.364478 29084 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:28:59.365367 29084 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:28:59.374462 29084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64dc09498a18477fb88987f736b8c3bd: Generated new TSK 0
I20260619 12:28:59.375164 29084 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:28:59.380396 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 28997
2026-06-19T12:28:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:28:59 dist-test-slave-kr9h krb5kdc[29096](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:28:59 dist-test-slave-kr9h krb5kdc[29096](info): set up 2 sockets
Jun 19 12:28:59 dist-test-slave-kr9h krb5kdc[29096](info): commencing operation
krb5kdc: starting...
W20260619 12:29:00.417906 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.018s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:29:00 dist-test-slave-kr9h krb5kdc[29096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:29:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:29:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:29:00.549752 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:37147
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:43855
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:37147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:29:00.666256 29109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:29:00.666560 29109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:29:00.666604 29109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:29:00.671145 29109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:29:00.671234 29109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:29:00.671257 29109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:29:00.671273 29109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:29:00.671289 29109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:29:00.676927 29109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:43855
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:37147
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:37147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:29:00.678175 29109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:29:00.679286 29109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:29:00.689266 29117 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:29:00.689220 29114 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:29:00.689332 29109 server_base.cc:1034] running on GCE node
W20260619 12:29:00.689224 29115 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:29:00.690263 29109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:29:00.691473 29109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:29:00.692680 29109 hybrid_clock.cc:648] HybridClock initialized: now 1781872140692649 us; error 41 us; skew 500 ppm
Jun 19 12:29:00 dist-test-slave-kr9h krb5kdc[29096](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872140, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:29:00.695978 29109 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:29:00.697126 29109 webserver.cc:460] Webserver started at http://127.6.231.254:35549/ using document root <none> and password file <none>
I20260619 12:29:00.697809 29109 fs_manager.cc:359] Metadata directory not provided
I20260619 12:29:00.697911 29109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:29:00.698165 29109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:29:00.700922 29109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "69a188bf8108471f9f2b153a8e06cf37"
format_stamp: "Formatted at 2026-06-19 12:29:00 on dist-test-slave-kr9h"
server_key: "761dd16c34769434827d6be19c014641"
server_key_iv: "a5a76baa1b980112c76d1835a9254be5"
server_key_version: "encryptionkey@0"
I20260619 12:29:00.701678 29109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "69a188bf8108471f9f2b153a8e06cf37"
format_stamp: "Formatted at 2026-06-19 12:29:00 on dist-test-slave-kr9h"
server_key: "761dd16c34769434827d6be19c014641"
server_key_iv: "a5a76baa1b980112c76d1835a9254be5"
server_key_version: "encryptionkey@0"
I20260619 12:29:00.706365 29109 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260619 12:29:00.709609 29124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:29:00.711027 29109 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260619 12:29:00.711194 29109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "69a188bf8108471f9f2b153a8e06cf37"
format_stamp: "Formatted at 2026-06-19 12:29:00 on dist-test-slave-kr9h"
server_key: "761dd16c34769434827d6be19c014641"
server_key_iv: "a5a76baa1b980112c76d1835a9254be5"
server_key_version: "encryptionkey@0"
I20260619 12:29:00.711330 29109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:29:00.732511 29109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:29:00.736315 29109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:29:00.736554 29109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:29:00.745008 29109 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:37147
I20260619 12:29:00.745036 29176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:37147 every 8 connection(s)
I20260619 12:29:00.746104 29109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:29:00.747411 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 29109
I20260619 12:29:00.747573 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:29:00.747864 7071 external_mini_cluster.cc:1428] Setting key 5c37fb461e5cbe1ea85741cbb62b6c6b
I20260619 12:29:00.749274 29177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:29:00.756603 29177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37: Bootstrap starting.
I20260619 12:29:00.758646 29180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:00.749227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42571 (local address 127.6.231.254:37147)
0619 12:29:00.749818 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:00.749839 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:00.749883 (+ 44us) server_negotiation.cc:407] Connection header received
0619 12:29:00.750717 (+ 834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:00.750739 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:00.751186 (+ 447us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:00.751564 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:00.751993 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:00.753557 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:00.754540 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:00.755258 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:00.755418 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:00.756421 (+ 1003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:00.756428 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:00.756445 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:29:00.756961 (+ 516us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:00.757032 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:29:00.757062 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:00.757172 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:00.757398 (+ 226us) server_negotiation.cc:299] Negotiation successful
0619 12:29:00.757654 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":173,"threads_started":1}
I20260619 12:29:00.759397 29177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37: Neither blocks nor log segments found. Creating new log.
I20260619 12:29:00.760262 29177 log.cc:824] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37: Log is configured to *not* fsync() on all Append() calls
I20260619 12:29:00.762099 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:29:00.762605 29177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37: No bootstrap required, opened a new log
I20260619 12:29:00.767026 29177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a188bf8108471f9f2b153a8e06cf37" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37147 } }
I20260619 12:29:00.767339 29177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:29:00.767395 29177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69a188bf8108471f9f2b153a8e06cf37, State: Initialized, Role: FOLLOWER
I20260619 12:29:00.767922 29177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a188bf8108471f9f2b153a8e06cf37" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37147 } }
I20260619 12:29:00.768112 29177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:29:00.768195 29177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:29:00.768289 29177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:29:00.768946 29180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:00.764418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39561 (local address 127.6.231.254:37147)
0619 12:29:00.764557 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:00.764562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:00.764721 (+ 159us) server_negotiation.cc:407] Connection header received
0619 12:29:00.764842 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:00.764846 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:00.764894 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:00.764983 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:00.765261 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:00.766161 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:00.767045 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:00.767392 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:00.767505 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:00.768523 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:00.768529 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:00.768535 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:29:00.768552 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:00.768581 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:29:00.768593 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:00.768669 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:00.768756 (+ 87us) server_negotiation.cc:299] Negotiation successful
0619 12:29:00.768817 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260619 12:29:00.769608 29177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a188bf8108471f9f2b153a8e06cf37" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37147 } }
I20260619 12:29:00.769994 29177 leader_election.cc:302] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69a188bf8108471f9f2b153a8e06cf37; no voters:
I20260619 12:29:00.770383 29177 leader_election.cc:288] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:29:00.770516 29187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:29:00.770809 29187 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [term 1 LEADER]: Becoming Leader. State: Replica: 69a188bf8108471f9f2b153a8e06cf37, State: Running, Role: LEADER
I20260619 12:29:00.771178 29187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a188bf8108471f9f2b153a8e06cf37" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37147 } }
I20260619 12:29:00.771687 29177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:29:00.773051 29189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69a188bf8108471f9f2b153a8e06cf37. Latest consensus state: current_term: 1 leader_uuid: "69a188bf8108471f9f2b153a8e06cf37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a188bf8108471f9f2b153a8e06cf37" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37147 } } }
I20260619 12:29:00.773108 29188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69a188bf8108471f9f2b153a8e06cf37" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69a188bf8108471f9f2b153a8e06cf37" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 37147 } } }
I20260619 12:29:00.773253 29189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [sys.catalog]: This master's current role is: LEADER
I20260619 12:29:00.773259 29188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37 [sys.catalog]: This master's current role is: LEADER
W20260619 12:29:00.776405 29202 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:29:00.776474 29202 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:29:00.776803 29201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:29:00.777594 29201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:29:00.783829 29201 catalog_manager.cc:1347] Generated new cluster ID: eae5e362a21c441596232af21e552c1d
I20260619 12:29:00.783912 29201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:29:00.791381 29201 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:29:00.792784 29201 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:29:00.801774 29201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69a188bf8108471f9f2b153a8e06cf37: Generated new TSK 0
I20260619 12:29:00.802747 29201 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:29:00.808756 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 29109
2026-06-19T12:29:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1428 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:29:00 dist-test-slave-kr9h krb5kdc[29208](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:29:00 dist-test-slave-kr9h krb5kdc[29208](info): set up 2 sockets
Jun 19 12:29:00 dist-test-slave-kr9h krb5kdc[29208](info): commencing operation
krb5kdc: starting...
W20260619 12:29:01.861877 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:29:01 dist-test-slave-kr9h krb5kdc[29208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872141, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:29:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:29:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:29:01.989715 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:39729
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:39271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:39729
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:29:02.104040 29221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:29:02.104375 29221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:29:02.104451 29221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:29:02.108635 29221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:29:02.108748 29221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:29:02.108783 29221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:29:02.108824 29221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:29:02.108855 29221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:29:02.114627 29221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:39271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:39729
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:39729
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:29:02.115809 29221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:29:02.116992 29221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:29:02.125932 29226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:29:02.125921 29229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:29:02.125922 29227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:29:02.126392 29221 server_base.cc:1034] running on GCE node
I20260619 12:29:02.126951 29221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:29:02.128122 29221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:29:02.129319 29221 hybrid_clock.cc:648] HybridClock initialized: now 1781872142129305 us; error 26 us; skew 500 ppm
Jun 19 12:29:02 dist-test-slave-kr9h krb5kdc[29208](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872142, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:29:02.132200 29221 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:29:02.133227 29221 webserver.cc:460] Webserver started at http://127.6.231.254:36035/ using document root <none> and password file <none>
I20260619 12:29:02.133862 29221 fs_manager.cc:359] Metadata directory not provided
I20260619 12:29:02.133955 29221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:29:02.134186 29221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:29:02.136302 29221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "01a2ff91482f41aab2f7b459e285e41d"
format_stamp: "Formatted at 2026-06-19 12:29:02 on dist-test-slave-kr9h"
server_key: "2d5c16fd94ebee6ebea76ada42601470"
server_key_iv: "85b2c138e296dcf3cfd70e87e45fe7b0"
server_key_version: "encryptionkey@0"
I20260619 12:29:02.136893 29221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "01a2ff91482f41aab2f7b459e285e41d"
format_stamp: "Formatted at 2026-06-19 12:29:02 on dist-test-slave-kr9h"
server_key: "2d5c16fd94ebee6ebea76ada42601470"
server_key_iv: "85b2c138e296dcf3cfd70e87e45fe7b0"
server_key_version: "encryptionkey@0"
I20260619 12:29:02.140877 29221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260619 12:29:02.143443 29236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:29:02.144752 29221 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:29:02.144899 29221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "01a2ff91482f41aab2f7b459e285e41d"
format_stamp: "Formatted at 2026-06-19 12:29:02 on dist-test-slave-kr9h"
server_key: "2d5c16fd94ebee6ebea76ada42601470"
server_key_iv: "85b2c138e296dcf3cfd70e87e45fe7b0"
server_key_version: "encryptionkey@0"
I20260619 12:29:02.145035 29221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:29:02.149819 29221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:29:02.153069 29221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:29:02.153270 29221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:29:02.160108 29221 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:39729
I20260619 12:29:02.160184 29288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:39729 every 8 connection(s)
I20260619 12:29:02.161162 29221 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:29:02.164017 29289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:29:02.166313 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 29221
I20260619 12:29:02.166451 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:29:02.166733 7071 external_mini_cluster.cc:1428] Setting key 07763cd7bec1c444948d40f0684a3e5a
I20260619 12:29:02.170743 29289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d: Bootstrap starting.
I20260619 12:29:02.172614 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:02.168043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33315 (local address 127.6.231.254:39729)
0619 12:29:02.168466 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:02.168486 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:02.168528 (+ 42us) server_negotiation.cc:407] Connection header received
0619 12:29:02.169583 (+ 1055us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:02.169607 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:02.169983 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:02.170363 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:02.171188 (+ 825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:02.171194 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:02.171209 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:29:02.171221 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:02.171283 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:29:02.171308 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:02.171447 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:02.171515 (+ 68us) server_negotiation.cc:299] Negotiation successful
0619 12:29:02.171661 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":99,"threads_started":1}
I20260619 12:29:02.173578 29289 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d: Neither blocks nor log segments found. Creating new log.
I20260619 12:29:02.174387 29289 log.cc:824] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d: Log is configured to *not* fsync() on all Append() calls
I20260619 12:29:02.175522 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:29:02.176364 29289 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d: No bootstrap required, opened a new log
I20260619 12:29:02.179401 29292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:02.177597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34973 (local address 127.6.231.254:39729)
0619 12:29:02.177798 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:02.177803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:02.177960 (+ 157us) server_negotiation.cc:407] Connection header received
0619 12:29:02.178125 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:02.178127 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:02.178165 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:02.178258 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:02.178990 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:02.178993 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:02.178996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0619 12:29:02.179008 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:02.179033 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0619 12:29:02.179052 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:02.179145 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:02.179238 (+ 93us) server_negotiation.cc:299] Negotiation successful
0619 12:29:02.179279 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260619 12:29:02.180797 29289 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a2ff91482f41aab2f7b459e285e41d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 39729 } }
I20260619 12:29:02.181025 29289 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260619 12:29:02.181108 29289 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01a2ff91482f41aab2f7b459e285e41d, State: Initialized, Role: FOLLOWER
I20260619 12:29:02.181612 29289 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a2ff91482f41aab2f7b459e285e41d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 39729 } }
I20260619 12:29:02.181764 29289 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260619 12:29:02.181855 29289 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260619 12:29:02.181978 29289 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:29:02.182921 29289 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a2ff91482f41aab2f7b459e285e41d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 39729 } }
I20260619 12:29:02.183303 29289 leader_election.cc:302] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01a2ff91482f41aab2f7b459e285e41d; no voters:
I20260619 12:29:02.183630 29289 leader_election.cc:288] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260619 12:29:02.183805 29299 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:29:02.184052 29299 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [term 1 LEADER]: Becoming Leader. State: Replica: 01a2ff91482f41aab2f7b459e285e41d, State: Running, Role: LEADER
I20260619 12:29:02.184357 29299 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a2ff91482f41aab2f7b459e285e41d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 39729 } }
I20260619 12:29:02.185000 29289 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:29:02.186096 29300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01a2ff91482f41aab2f7b459e285e41d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a2ff91482f41aab2f7b459e285e41d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 39729 } } }
I20260619 12:29:02.186230 29300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [sys.catalog]: This master's current role is: LEADER
I20260619 12:29:02.187019 29301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01a2ff91482f41aab2f7b459e285e41d. Latest consensus state: current_term: 1 leader_uuid: "01a2ff91482f41aab2f7b459e285e41d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01a2ff91482f41aab2f7b459e285e41d" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 39729 } } }
I20260619 12:29:02.187212 29301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d [sys.catalog]: This master's current role is: LEADER
W20260619 12:29:02.190193 29314 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:29:02.190264 29314 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:29:02.190342 29308 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:29:02.191094 29308 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:29:02.197150 29308 catalog_manager.cc:1347] Generated new cluster ID: feeed2aee2e94d9a99f8238bf5c896f4
I20260619 12:29:02.197235 29308 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:29:02.213418 29308 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:29:02.213843 29308 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:29:02.221395 29308 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01a2ff91482f41aab2f7b459e285e41d: Generated new TSK 0
I20260619 12:29:02.222359 29308 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260619 12:29:02.236004 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 29221
2026-06-19T12:29:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1426 ms)
[----------] 18 tests from AuthTokenIssuingTest (26084 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:29:02 dist-test-slave-kr9h krb5kdc[29320](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:29:02 dist-test-slave-kr9h krb5kdc[29320](info): set up 2 sockets
Jun 19 12:29:02 dist-test-slave-kr9h krb5kdc[29320](info): commencing operation
krb5kdc: starting...
W20260619 12:29:03.290036 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:29:03 dist-test-slave-kr9h krb5kdc[29320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:29:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:29:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:29:03.412779 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:46407
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:45871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:46407,127.6.231.253:43905,127.6.231.252:35775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:29:03.535969 29333 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:29:03.536378 29333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:29:03.536454 29333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:29:03.536543 29333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:29:03.541219 29333 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:29:03.541328 29333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:29:03.541371 29333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:29:03.541432 29333 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:29:03.541508 29333 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:29:03.547610 29333 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:45871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:46407,127.6.231.253:43905,127.6.231.252:35775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:46407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:29:03.548857 29333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:29:03.550026 29333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:29:03.559584 29338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:29:03.559649 29333 server_base.cc:1034] running on GCE node
W20260619 12:29:03.559587 29339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:29:03.559587 29341 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:29:03.560560 29333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:29:03.561918 29333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:29:03.563126 29333 hybrid_clock.cc:648] HybridClock initialized: now 1781872143563097 us; error 45 us; skew 500 ppm
Jun 19 12:29:03 dist-test-slave-kr9h krb5kdc[29320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:29:03.571332 29333 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:29:03.572348 29333 webserver.cc:460] Webserver started at http://127.6.231.254:42761/ using document root <none> and password file <none>
I20260619 12:29:03.572983 29333 fs_manager.cc:359] Metadata directory not provided
I20260619 12:29:03.573052 29333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:29:03.573243 29333 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:29:03.575537 29333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "cc931d011b184f718a1fb50e9eacc9e3"
format_stamp: "Formatted at 2026-06-19 12:29:03 on dist-test-slave-kr9h"
server_key: "c359309bba2e988be55a38bc4ab72b41"
server_key_iv: "5274ca9a06ecde31107e4e1b6ae35273"
server_key_version: "encryptionkey@0"
I20260619 12:29:03.576100 29333 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "cc931d011b184f718a1fb50e9eacc9e3"
format_stamp: "Formatted at 2026-06-19 12:29:03 on dist-test-slave-kr9h"
server_key: "c359309bba2e988be55a38bc4ab72b41"
server_key_iv: "5274ca9a06ecde31107e4e1b6ae35273"
server_key_version: "encryptionkey@0"
I20260619 12:29:03.580217 29333 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260619 12:29:03.582826 29348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:29:03.584139 29333 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260619 12:29:03.584280 29333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "cc931d011b184f718a1fb50e9eacc9e3"
format_stamp: "Formatted at 2026-06-19 12:29:03 on dist-test-slave-kr9h"
server_key: "c359309bba2e988be55a38bc4ab72b41"
server_key_iv: "5274ca9a06ecde31107e4e1b6ae35273"
server_key_version: "encryptionkey@0"
I20260619 12:29:03.584391 29333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:29:03.600749 29333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:29:03.603889 29333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:29:03.604110 29333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:29:03.611299 29333 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:46407
I20260619 12:29:03.611584 29400 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:46407 every 8 connection(s)
I20260619 12:29:03.612313 29333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:29:03.615181 29401 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:29:03.618913 29401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:03.620038 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 29333
I20260619 12:29:03.620182 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:29:03.620494 7071 external_mini_cluster.cc:1428] Setting key e9731ab19004b2a1cf701296609d016b
Jun 19 12:29:03 dist-test-slave-kr9h krb5kdc[29320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.6.231.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.253@KRBTEST.COM" created.
I20260619 12:29:03.634929 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.621372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:46407 (local address 127.0.0.1:56118)
0619 12:29:03.621982 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.622018 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:03.623283 (+ 1265us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:03.624032 (+ 749us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:03.624037 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:03.624378 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:03.624677 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:03.624698 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.625931 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.625934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:03.626684 (+ 750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:03.626702 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.627350 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.627354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:03.627461 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:03.628052 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:03.628069 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:03.629739 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:03.631850 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:03.631858 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:03.631867 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:03.632174 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:03.632537 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:03.632540 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:03.632541 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:03.632712 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:03.633357 (+ 645us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:03.633367 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:03.633569 (+ 202us) client_negotiation.cc:769] Sending connection context
0619 12:29:03.633817 (+ 248us) client_negotiation.cc:240] Negotiation successful
0619 12:29:03.634113 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":443,"spinlock_wait_cycles":20864,"thread_start_us":153,"threads_started":1}
I20260619 12:29:03.634935 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.621371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56118 (local address 127.6.231.254:46407)
0619 12:29:03.621824 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:03.621843 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:03.622122 (+ 279us) server_negotiation.cc:407] Connection header received
0619 12:29:03.623643 (+ 1521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:03.623648 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:03.623844 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:03.624000 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:03.624844 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.625816 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.626827 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.627200 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.627366 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:03.629905 (+ 2539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:03.629911 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:03.629923 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:03.629945 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:03.631742 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:03.632284 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:03.632293 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:03.632298 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:03.632442 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:03.632806 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:03.632808 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:03.632809 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:03.633240 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:03.633352 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:03.633930 (+ 578us) server_negotiation.cc:299] Negotiation successful
0619 12:29:03.634245 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":90,"server-negotiator.queue_time_us":309,"thread_start_us":136,"threads_started":1}
I20260619 12:29:03.637353 29401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:03.638104 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.637856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:43905 (local address 127.0.0.1:49484)
0619 12:29:03.637974 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.638046 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:43905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 12:29:03.638275 29349 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.231.253:43905: connect: Connection refused (error 111)
I20260619 12:29:03.639047 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.638851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:43905 (local address 127.0.0.1:49490)
0619 12:29:03.638949 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.638998 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:43905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 12:29:03.639195 29401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.253:43905: Network error: Client connection negotiation failed: client connection to 127.6.231.253:43905: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.253.keytab.
Entry for principal kudu/127.6.231.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.253.keytab.
I20260619 12:29:03.648854 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.253:43905
--webserver_interface=127.6.231.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.253.keytab
--principal=kudu/127.6.231.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:45871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:46407,127.6.231.253:43905,127.6.231.252:35775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:29:03.688465 29401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } attempt: 1
I20260619 12:29:03.689411 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.689038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:43905 (local address 127.0.0.1:49492)
0619 12:29:03.689225 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.689316 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:43905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260619 12:29:03.690208 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.689986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:43905 (local address 127.0.0.1:49506)
0619 12:29:03.690111 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.690158 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:43905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 12:29:03.690418 29401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.253:43905: Network error: Client connection negotiation failed: client connection to 127.6.231.253:43905: connect: Connection refused (error 111)
I20260619 12:29:03.758605 29401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } attempt: 2
I20260619 12:29:03.759483 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.759181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:43905 (local address 127.0.0.1:49514)
0619 12:29:03.759341 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.759407 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:43905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260619 12:29:03.760315 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.760111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:43905 (local address 127.0.0.1:49520)
0619 12:29:03.760216 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.760262 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:43905: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260619 12:29:03.760507 29401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.253:43905: Network error: Client connection negotiation failed: client connection to 127.6.231.253:43905: connect: Connection refused (error 111)
W20260619 12:29:03.763073 29408 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:29:03.763383 29408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:29:03.763447 29408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:29:03.763533 29408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:29:03.767958 29408 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:29:03.768038 29408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:29:03.768057 29408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:29:03.768074 29408 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:29:03.768090 29408 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:29:03.773891 29408 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:45871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:46407,127.6.231.253:43905,127.6.231.252:35775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.253.keytab
--principal=kudu/127.6.231.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.253:43905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.231.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:29:03.775069 29408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:29:03.776129 29408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:29:03.785094 29414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:29:03.785094 29413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:29:03.785094 29416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:29:03.785530 29408 server_base.cc:1034] running on GCE node
I20260619 12:29:03.786126 29408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:29:03.787303 29408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:29:03.788479 29408 hybrid_clock.cc:648] HybridClock initialized: now 1781872143788431 us; error 64 us; skew 500 ppm
Jun 19 12:29:03 dist-test-slave-kr9h krb5kdc[29320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:29:03.791383 29408 init.cc:377] Logged in from keytab as kudu/127.6.231.253@KRBTEST.COM (short username kudu)
I20260619 12:29:03.792296 29408 webserver.cc:460] Webserver started at http://127.6.231.253:35241/ using document root <none> and password file <none>
I20260619 12:29:03.792842 29408 fs_manager.cc:359] Metadata directory not provided
I20260619 12:29:03.792927 29408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:29:03.793149 29408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:29:03.795185 29408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/data/instance:
uuid: "5167651f0a8042bb89859bee031786c1"
format_stamp: "Formatted at 2026-06-19 12:29:03 on dist-test-slave-kr9h"
server_key: "93acc79e6ae4b91db81f35f39c041bbd"
server_key_iv: "cb3868ed8efd0b9a9a5a74bbd73cd4aa"
server_key_version: "encryptionkey@0"
I20260619 12:29:03.795755 29408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/wal/instance:
uuid: "5167651f0a8042bb89859bee031786c1"
format_stamp: "Formatted at 2026-06-19 12:29:03 on dist-test-slave-kr9h"
server_key: "93acc79e6ae4b91db81f35f39c041bbd"
server_key_iv: "cb3868ed8efd0b9a9a5a74bbd73cd4aa"
server_key_version: "encryptionkey@0"
I20260619 12:29:03.799485 29408 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260619 12:29:03.801944 29423 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:29:03.803102 29408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:29:03.803254 29408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/wal
uuid: "5167651f0a8042bb89859bee031786c1"
format_stamp: "Formatted at 2026-06-19 12:29:03 on dist-test-slave-kr9h"
server_key: "93acc79e6ae4b91db81f35f39c041bbd"
server_key_iv: "cb3868ed8efd0b9a9a5a74bbd73cd4aa"
server_key_version: "encryptionkey@0"
I20260619 12:29:03.803367 29408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:29:03.832434 29408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:29:03.836033 29408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:29:03.836290 29408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:29:03.844024 29408 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.253:43905
I20260619 12:29:03.844338 29475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.253:43905 every 8 connection(s)
I20260619 12:29:03.845101 29408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/data/info.pb
I20260619 12:29:03.845577 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 29408
I20260619 12:29:03.845726 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-1/wal/instance
I20260619 12:29:03.846055 7071 external_mini_cluster.cc:1428] Setting key b986edb440ce933792351fd9b62e3197
I20260619 12:29:03.848312 29476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:29:03.851691 29401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } attempt: 3
I20260619 12:29:03.852530 29476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.231.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.252@KRBTEST.COM" created.
Jun 19 12:29:03 dist-test-slave-kr9h krb5kdc[29320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.253@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
Jun 19 12:29:03 dist-test-slave-kr9h krb5kdc[29320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.253@KRBTEST.COM
I20260619 12:29:03.865724 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.852302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:43905 (local address 127.0.0.1:49528)
0619 12:29:03.852552 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.852571 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:03.852675 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:03.855389 (+ 2714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:03.855393 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:03.855593 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:03.855701 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:03.855708 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.857478 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.857482 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:03.858431 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:03.858447 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.859284 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.859289 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:03.859409 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:03.860008 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:03.860029 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:03.861893 (+ 1864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:03.863937 (+ 2044us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:03.863940 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:03.863942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:03.864151 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:03.864488 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:03.864494 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:03.864498 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:03.864588 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:03.865178 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:03.865181 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:03.865280 (+ 99us) client_negotiation.cc:769] Sending connection context
0619 12:29:03.865389 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 12:29:03.865556 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144}
I20260619 12:29:03.866506 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.854640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56122 (local address 127.6.231.254:46407)
0619 12:29:03.854770 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:03.854775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:03.855069 (+ 294us) server_negotiation.cc:407] Connection header received
0619 12:29:03.855389 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:03.855393 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:03.855456 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:03.855548 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:03.856458 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.857312 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.858356 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.858779 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.858896 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:03.861790 (+ 2894us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:03.861796 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:03.861803 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:03.861836 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:03.863687 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:03.864293 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:03.864296 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:03.864298 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:03.864385 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:03.864789 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:03.864794 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:03.864798 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:03.865039 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:03.865122 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:03.866306 (+ 1184us) server_negotiation.cc:299] Negotiation successful
0619 12:29:03.866403 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260619 12:29:03.866948 29479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.853166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49528 (local address 127.6.231.253:43905)
0619 12:29:03.853715 (+ 549us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:03.853734 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:03.853777 (+ 43us) server_negotiation.cc:407] Connection header received
0619 12:29:03.854609 (+ 832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:03.854628 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:03.855162 (+ 534us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:03.855448 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:03.855874 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.857323 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.858608 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.859165 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.859264 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:03.862033 (+ 2769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:03.862044 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:03.862056 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:03.862097 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:03.863801 (+ 1704us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:03.864261 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:03.864267 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:03.864269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:03.864376 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:03.864676 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:03.864680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:03.864682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:03.865063 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:03.865200 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:03.865545 (+ 345us) server_negotiation.cc:299] Negotiation successful
0619 12:29:03.865883 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":397,"thread_start_us":195,"threads_started":1}
I20260619 12:29:03.867000 29480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.854563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:46407 (local address 127.0.0.1:56122)
0619 12:29:03.854924 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.854977 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:03.855146 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:03.855596 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:03.855601 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:03.856055 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:03.856294 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:03.856308 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.857463 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.857476 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:03.858225 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:03.858234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.858937 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.858941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:03.859155 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:03.859851 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:03.859877 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:03.861644 (+ 1767us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:03.863823 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:03.863832 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:03.863844 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:03.864166 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:03.864516 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:03.864519 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:03.864521 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:03.864666 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:03.865217 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:03.865227 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:03.865400 (+ 173us) client_negotiation.cc:769] Sending connection context
0619 12:29:03.866650 (+ 1250us) client_negotiation.cc:240] Negotiation successful
0619 12:29:03.866871 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":26,"thread_start_us":110,"threads_started":1}
I20260619 12:29:03.868824 29476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:03.869009 29401 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:03.869683 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.869414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42600)
0619 12:29:03.869562 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.869617 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 12:29:03.870355 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.870068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42606)
0619 12:29:03.870231 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.870282 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260619 12:29:03.870472 29401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:35775: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
Jun 19 12:29:03 dist-test-slave-kr9h krb5kdc[29320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.253@KRBTEST.COM for kudu/127.6.231.253@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.252.keytab.
Entry for principal kudu/127.6.231.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.252.keytab.
I20260619 12:29:03.876396 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.252:35775
--webserver_interface=127.6.231.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.252.keytab
--principal=kudu/127.6.231.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:45871
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:46407,127.6.231.253:43905,127.6.231.252:35775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:29:03.877920 29479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.869618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49540 (local address 127.6.231.253:43905)
0619 12:29:03.869708 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:03.869712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:03.869724 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:29:03.869852 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:03.869854 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:03.869890 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:03.869969 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:03.870382 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.871072 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.872073 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.872360 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.872443 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:03.874643 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:03.874647 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:03.874650 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:03.874672 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:03.876247 (+ 1575us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:03.876722 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:03.876727 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:03.876729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:03.876800 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:03.877064 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:03.877067 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:03.877069 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:03.877246 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:03.877340 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:03.877631 (+ 291us) server_negotiation.cc:299] Negotiation successful
0619 12:29:03.877793 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":68,"server-negotiator.queue_time_us":38}
I20260619 12:29:03.877920 29480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.869500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:43905 (local address 127.0.0.1:49540)
0619 12:29:03.869598 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.869611 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:03.869710 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:03.869979 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:03.869981 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:03.870148 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:03.870263 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:03.870271 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.871211 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.871213 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:03.871945 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:03.871954 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:03.872461 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:03.872464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:03.872566 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:03.873013 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:03.873023 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:03.874457 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:03.876363 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:03.876366 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:03.876368 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:03.876642 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:03.876885 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:03.876890 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:03.876892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:03.876975 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:03.877339 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:03.877342 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:03.877475 (+ 133us) client_negotiation.cc:769] Sending connection context
0619 12:29:03.877589 (+ 114us) client_negotiation.cc:240] Negotiation successful
0619 12:29:03.877736 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260619 12:29:03.878970 29476 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:03.879678 29480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.879425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42612)
0619 12:29:03.879536 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.879620 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 12:29:03.879933 29424 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
I20260619 12:29:03.880867 29480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.880670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42616)
0619 12:29:03.880778 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.880818 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260619 12:29:03.881095 29476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:35775: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
I20260619 12:29:03.901700 29401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } attempt: 1
I20260619 12:29:03.902652 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.902342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42620)
0619 12:29:03.902514 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.902576 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260619 12:29:03.903369 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.903183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42622)
0619 12:29:03.903277 (+ 94us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.903319 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 12:29:03.903505 29401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:35775: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
I20260619 12:29:03.930374 29476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } attempt: 1
I20260619 12:29:03.931391 29480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.931082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42630)
0619 12:29:03.931248 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.931312 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260619 12:29:03.932524 29480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.932242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42642)
0619 12:29:03.932365 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.932449 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260619 12:29:03.932732 29476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:35775: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
I20260619 12:29:03.978673 29401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } attempt: 2
I20260619 12:29:03.979604 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.979254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42656)
0619 12:29:03.979437 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.979513 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260619 12:29:03.980436 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:03.980226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42670)
0619 12:29:03.980338 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:03.980384 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260619 12:29:03.980721 29401 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:35775: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
W20260619 12:29:03.992218 29482 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:29:03.992553 29482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:29:03.992612 29482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:29:03.992687 29482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:29:03.996759 29482 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:29:03.996843 29482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:29:03.996878 29482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:29:03.996909 29482 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:29:03.996946 29482 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:29:04.000885 29476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } attempt: 2
I20260619 12:29:04.001799 29480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.001448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42682)
0619 12:29:04.001653 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:04.001715 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260619 12:29:04.002554 29480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.002338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42698)
0619 12:29:04.002443 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:04.002499 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 12:29:04.002669 29476 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:35775: Network error: Client connection negotiation failed: client connection to 127.6.231.252:35775: connect: Connection refused (error 111)
I20260619 12:29:04.002709 29482 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:45871
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:46407,127.6.231.253:43905,127.6.231.252:35775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.252.keytab
--principal=kudu/127.6.231.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.252:35775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.231.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:29:04.003844 29482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:29:04.004935 29482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:29:04.014213 29488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:29:04.014335 29491 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:29:04.014213 29489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:29:04.014614 29482 server_base.cc:1034] running on GCE node
I20260619 12:29:04.015193 29482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:29:04.016376 29482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:29:04.017517 29482 hybrid_clock.cc:648] HybridClock initialized: now 1781872144017474 us; error 51 us; skew 500 ppm
Jun 19 12:29:04 dist-test-slave-kr9h krb5kdc[29320](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872144, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:29:04.020381 29482 init.cc:377] Logged in from keytab as kudu/127.6.231.252@KRBTEST.COM (short username kudu)
I20260619 12:29:04.021354 29482 webserver.cc:460] Webserver started at http://127.6.231.252:43895/ using document root <none> and password file <none>
I20260619 12:29:04.021936 29482 fs_manager.cc:359] Metadata directory not provided
I20260619 12:29:04.022035 29482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:29:04.022267 29482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:29:04.024356 29482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/data/instance:
uuid: "571b0adbe6054726a89c9e9b767c6da1"
format_stamp: "Formatted at 2026-06-19 12:29:04 on dist-test-slave-kr9h"
server_key: "a2abdcd2872a4bd4e70c0965d217d204"
server_key_iv: "2e0814145c1471d3db21c9ae60b51de2"
server_key_version: "encryptionkey@0"
I20260619 12:29:04.024935 29482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/wal/instance:
uuid: "571b0adbe6054726a89c9e9b767c6da1"
format_stamp: "Formatted at 2026-06-19 12:29:04 on dist-test-slave-kr9h"
server_key: "a2abdcd2872a4bd4e70c0965d217d204"
server_key_iv: "2e0814145c1471d3db21c9ae60b51de2"
server_key_version: "encryptionkey@0"
I20260619 12:29:04.029049 29482 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260619 12:29:04.031539 29498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:29:04.032814 29482 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260619 12:29:04.032940 29482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/wal
uuid: "571b0adbe6054726a89c9e9b767c6da1"
format_stamp: "Formatted at 2026-06-19 12:29:04 on dist-test-slave-kr9h"
server_key: "a2abdcd2872a4bd4e70c0965d217d204"
server_key_iv: "2e0814145c1471d3db21c9ae60b51de2"
server_key_version: "encryptionkey@0"
I20260619 12:29:04.033043 29482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:29:04.060114 29482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:29:04.063582 29482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:29:04.063792 29482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:29:04.071135 29482 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.252:35775
I20260619 12:29:04.071421 29550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.252:35775 every 8 connection(s)
I20260619 12:29:04.072101 29482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/data/info.pb
I20260619 12:29:04.073343 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 29482
I20260619 12:29:04.073503 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1781871857195028-7071-0/minicluster-data/master-2/wal/instance
I20260619 12:29:04.073808 7071 external_mini_cluster.cc:1428] Setting key 8881f6f8ad0061fecd26234ff83df82e
I20260619 12:29:04.075323 29551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:29:04.079900 29401 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } attempt: 3
I20260619 12:29:04.080116 29551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } has no permanent_uuid. Determining permanent_uuid...
Jun 19 12:29:04 dist-test-slave-kr9h krb5kdc[29320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.252@KRBTEST.COM
Jun 19 12:29:04 dist-test-slave-kr9h krb5kdc[29320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.252@KRBTEST.COM
I20260619 12:29:04.089332 29404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.080368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42726)
0619 12:29:04.080503 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:04.080520 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:04.080626 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:04.081038 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:04.081044 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:04.081227 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:04.081307 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:04.081312 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.082361 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.082365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:04.083159 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:04.083170 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.083748 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.083751 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:04.083846 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.084307 (+ 461us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:04.084318 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:04.085705 (+ 1387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:04.087650 (+ 1945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:04.087653 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:04.087656 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:04.087923 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:04.088216 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:04.088222 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:04.088224 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:04.088348 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:04.088745 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:04.088750 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:04.088869 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 12:29:04.088978 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 12:29:04.089124 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
Jun 19 12:29:04 dist-test-slave-kr9h krb5kdc[29320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872144, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.252@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:29:04.090325 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.080432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42726 (local address 127.6.231.252:35775)
0619 12:29:04.080786 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.080800 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.080815 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:29:04.080858 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.080861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.080922 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.081020 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.081412 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.082227 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.083316 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.083638 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.083747 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.085875 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:04.085878 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:04.085880 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:04.085899 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:04.087489 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.088002 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.088008 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.088011 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.088077 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.088450 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.088453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.088454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.088653 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:04.088747 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:04.089818 (+ 1071us) server_negotiation.cc:299] Negotiation successful
0619 12:29:04.089960 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":135,"threads_started":1}
I20260619 12:29:04.090353 29554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.075235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42712 (local address 127.6.231.252:35775)
0619 12:29:04.075653 (+ 418us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.075671 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.075715 (+ 44us) server_negotiation.cc:407] Connection header received
0619 12:29:04.076573 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.076595 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.077015 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.077371 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.078131 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.079658 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.080828 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.081530 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.081691 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.084870 (+ 3179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:04.084876 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:04.084887 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:04.084917 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:04.087074 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.087522 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.087530 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.087537 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.087675 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.087960 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.087963 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.087965 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.088437 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:04.088615 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:04.089036 (+ 421us) server_negotiation.cc:299] Negotiation successful
0619 12:29:04.089308 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":131,"threads_started":1}
I20260619 12:29:04.093806 29476 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } attempt: 3
I20260619 12:29:04.095196 29401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3: Bootstrap starting.
I20260619 12:29:04.095358 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.082433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:46407 (local address 127.0.0.1:56126)
0619 12:29:04.082825 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:04.082879 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:04.082997 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:04.083579 (+ 582us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:04.083586 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:04.084031 (+ 445us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:04.084176 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:04.084185 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.085125 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.085130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:04.085980 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:04.085994 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.086609 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.086613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:04.086914 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.087603 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:04.087618 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:04.090023 (+ 2405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:04.092607 (+ 2584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:04.092614 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:04.092619 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:04.092902 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:04.093192 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:04.093194 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:04.093196 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:04.093302 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:04.094699 (+ 1397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:04.094705 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:04.094820 (+ 115us) client_negotiation.cc:769] Sending connection context
0619 12:29:04.094975 (+ 155us) client_negotiation.cc:240] Negotiation successful
0619 12:29:04.095151 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":99,"threads_started":1}
I20260619 12:29:04.095808 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.083205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56126 (local address 127.6.231.254:46407)
0619 12:29:04.083332 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.083337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.083348 (+ 11us) server_negotiation.cc:407] Connection header received
0619 12:29:04.083376 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.083378 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.083413 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.083483 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.084323 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.085023 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.086118 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.086474 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.086604 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.090198 (+ 3594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:04.090202 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:04.090205 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:04.090229 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:04.092490 (+ 2261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.092997 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.093000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.093001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.093068 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.093455 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.093460 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.093462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.093863 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:04.094105 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:04.095078 (+ 973us) server_negotiation.cc:299] Negotiation successful
0619 12:29:04.095217 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260619 12:29:04.097867 29401 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3: Neither blocks nor log segments found. Creating new log.
I20260619 12:29:04.098467 29551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:04.098806 29401 log.cc:824] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3: Log is configured to *not* fsync() on all Append() calls
I20260619 12:29:04.101684 29401 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3: No bootstrap required, opened a new log
Jun 19 12:29:04 dist-test-slave-kr9h krb5kdc[29320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872143, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.253@KRBTEST.COM for kudu/127.6.231.252@KRBTEST.COM
Jun 19 12:29:04 dist-test-slave-kr9h krb5kdc[29320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:29:04.106246 29480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.094202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42742)
0619 12:29:04.094361 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:04.094376 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:04.094480 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:04.098130 (+ 3650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:04.098133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:04.098322 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:04.098432 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:04.098439 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.099527 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.099532 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:04.100238 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:04.100248 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.100704 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.100708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:04.100791 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.101261 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:04.101280 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:04.102582 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:04.104365 (+ 1783us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:04.104371 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:04.104373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:04.104889 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:04.105158 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:04.105162 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:04.105171 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:04.105255 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:04.105750 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:04.105753 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:04.105857 (+ 104us) client_negotiation.cc:769] Sending connection context
0619 12:29:04.105954 (+ 97us) client_negotiation.cc:240] Negotiation successful
0619 12:29:04.106080 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260619 12:29:04.106420 29401 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } }
I20260619 12:29:04.106745 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.094410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42742 (local address 127.6.231.252:35775)
0619 12:29:04.097842 (+ 3432us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.097846 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.097860 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:29:04.097898 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.097903 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.097991 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.098544 (+ 553us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.098675 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.099440 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.100359 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.100592 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.100704 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.102709 (+ 2005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:04.102712 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:04.102715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:04.102742 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:04.104269 (+ 1527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.104988 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.104993 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.104994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.105071 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.105341 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.105345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.105347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.105612 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:04.105708 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:04.106435 (+ 727us) server_negotiation.cc:299] Negotiation successful
0619 12:29:04.106573 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3365}
I20260619 12:29:04.106686 29401 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc931d011b184f718a1fb50e9eacc9e3, State: Initialized, Role: FOLLOWER
I20260619 12:29:04.107110 29401 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } }
I20260619 12:29:04.107717 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.097554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56130 (local address 127.6.231.254:46407)
0619 12:29:04.097664 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.097669 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.097686 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:29:04.097725 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.097730 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.097779 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.097863 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.098811 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.099502 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.100610 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.100912 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.100985 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.103814 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:04.103818 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:04.103820 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:04.103843 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:04.105512 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.106100 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.106104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.106106 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.106200 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.106502 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.106505 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.106506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.106757 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:04.106907 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:04.107320 (+ 413us) server_negotiation.cc:299] Negotiation successful
0619 12:29:04.107441 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260619 12:29:04.108629 29401 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:29:04.109858 29476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1: Bootstrap starting.
I20260619 12:29:04.112381 29476 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1: Neither blocks nor log segments found. Creating new log.
I20260619 12:29:04.113205 29476 log.cc:824] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1: Log is configured to *not* fsync() on all Append() calls
W20260619 12:29:04.113425 29569 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:29:04.113555 29569 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:29:04.115036 29558 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } } }
I20260619 12:29:04.115192 29558 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:04.114282 29333 master_runner.cc:186] Error getting master registration for 127.6.231.253:43905: OK, instance_id { permanent_uuid: "5167651f0a8042bb89859bee031786c1" instance_seqno: 1781872143841980 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:29:04.118188 29333 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 19 12:29:04 dist-test-slave-kr9h krb5kdc[29320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872143, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.253@KRBTEST.COM
Jun 19 12:29:04 dist-test-slave-kr9h krb5kdc[29320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872144, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.252@KRBTEST.COM for kudu/127.6.231.253@KRBTEST.COM
I20260619 12:29:04.122699 29476 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1: No bootstrap required, opened a new log
I20260619 12:29:04.124760 29571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.113589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49558 (local address 127.6.231.253:43905)
0619 12:29:04.113902 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.113910 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.113925 (+ 15us) server_negotiation.cc:407] Connection header received
0619 12:29:04.113960 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.113963 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.114004 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.114859 (+ 855us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.114978 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.115606 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.116361 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.116616 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.116716 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.119150 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:04.119154 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:04.119157 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:04.119180 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:04.122277 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.122810 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.122815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.122818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.122900 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.123285 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.123288 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.123289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.123509 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:04.124330 (+ 821us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:04.124394 (+ 64us) server_negotiation.cc:299] Negotiation successful
0619 12:29:04.124564 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":222,"thread_start_us":106,"threads_started":1}
I20260619 12:29:04.125074 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.098893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:43905 (local address 127.0.0.1:49542)
0619 12:29:04.099308 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:04.099321 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:04.099395 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:04.099748 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:04.099751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:04.099916 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:04.100016 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:04.100022 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.106669 (+ 6647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.106673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:04.114785 (+ 8112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:04.114802 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.118881 (+ 4079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.118885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:04.119045 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.119767 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:04.119787 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:04.121266 (+ 1479us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:04.123417 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:04.123420 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:04.123422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:04.123678 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:04.124025 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:04.124028 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:04.124030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:04.124121 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:04.124620 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:04.124623 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:04.124716 (+ 93us) client_negotiation.cc:769] Sending connection context
0619 12:29:04.124823 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 12:29:04.124963 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359}
I20260619 12:29:04.125960 29479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.099097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49542 (local address 127.6.231.253:43905)
0619 12:29:04.099222 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.099226 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.099392 (+ 166us) server_negotiation.cc:407] Connection header received
0619 12:29:04.099473 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.099476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.099520 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.099608 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.105569 (+ 5961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.106293 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.118287 (+ 11994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.118648 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.118752 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.121381 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:04.121385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:04.121388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:04.121412 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:04.123302 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.123831 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.123835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.123838 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.123920 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.124230 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.124234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.124235 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.124422 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:04.124575 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:04.125711 (+ 1136us) server_negotiation.cc:299] Negotiation successful
0619 12:29:04.125826 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":31,"server-negotiator.queue_time_us":39}
I20260619 12:29:04.126603 29551 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:04.127802 29476 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } }
I20260619 12:29:04.128031 29476 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5167651f0a8042bb89859bee031786c1, State: Initialized, Role: FOLLOWER
I20260619 12:29:04.128397 29476 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } }
I20260619 12:29:04.128629 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:29:04.129799 29476 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:29:04.129972 29375 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cc931d011b184f718a1fb50e9eacc9e3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56130
I20260619 12:29:04.130232 29375 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 12:29:04.130332 29375 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:29:04.131891 29375 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } }
Jun 19 12:29:04 dist-test-slave-kr9h krb5kdc[29320](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872144, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.252@KRBTEST.COM for kudu/127.6.231.252@KRBTEST.COM
I20260619 12:29:04.132903 29375 leader_election.cc:288] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [CANDIDATE]: Term 1 election: Requested vote from peers 5167651f0a8042bb89859bee031786c1 (127.6.231.253:43905), 571b0adbe6054726a89c9e9b767c6da1 (127.6.231.252:35775)
I20260619 12:29:04.133988 29450 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc931d011b184f718a1fb50e9eacc9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5167651f0a8042bb89859bee031786c1"
I20260619 12:29:04.134249 29450 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:29:04.135749 29450 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc931d011b184f718a1fb50e9eacc9e3 in term 1.
I20260619 12:29:04.136142 29349 leader_election.cc:302] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5167651f0a8042bb89859bee031786c1, cc931d011b184f718a1fb50e9eacc9e3; no voters:
I20260619 12:29:04.136420 29558 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:29:04.136652 29558 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [term 1 LEADER]: Becoming Leader. State: Replica: cc931d011b184f718a1fb50e9eacc9e3, State: Running, Role: LEADER
I20260619 12:29:04.136844 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.127125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42756 (local address 127.6.231.252:35775)
0619 12:29:04.127236 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.127241 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.127729 (+ 488us) server_negotiation.cc:407] Connection header received
0619 12:29:04.127767 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.127770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.127815 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.127884 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.128362 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.129102 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.130085 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.130381 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.130485 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.132957 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:04.132962 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:04.132965 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:04.132987 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:04.134786 (+ 1799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.135277 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.135280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.135282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.135362 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.135629 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.135634 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.135639 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.135849 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:04.135941 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:04.136595 (+ 654us) server_negotiation.cc:299] Negotiation successful
0619 12:29:04.136708 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 12:29:04.136843 29584 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:29:04.136907 29584 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:29:04.136986 29556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.127053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:35775 (local address 127.0.0.1:42756)
0619 12:29:04.127296 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:04.127310 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:04.127381 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:04.127980 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:04.127983 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:04.128162 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:04.128255 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:04.128262 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.129220 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.129223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:04.129965 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:04.129977 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.130612 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.130616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:04.130699 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.131199 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:04.131212 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:04.132822 (+ 1610us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:04.134929 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:04.134932 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:04.134934 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:04.135179 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:04.135450 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:04.135453 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:04.135455 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:04.135530 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:04.136011 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:04.136016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:04.136135 (+ 119us) client_negotiation.cc:769] Sending connection context
0619 12:29:04.136234 (+ 99us) client_negotiation.cc:240] Negotiation successful
0619 12:29:04.136372 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200}
I20260619 12:29:04.137001 29558 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } }
I20260619 12:29:04.137205 29573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } } }
I20260619 12:29:04.137311 29573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:04.138794 29558 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc931d011b184f718a1fb50e9eacc9e3. Latest consensus state: current_term: 1 leader_uuid: "cc931d011b184f718a1fb50e9eacc9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } } }
I20260619 12:29:04.138887 29558 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [sys.catalog]: This master's current role is: LEADER
I20260619 12:29:04.139771 29408 master_runner.cc:186] Error getting master registration for 127.6.231.252:35775: OK, instance_id { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" instance_seqno: 1781872144069149 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:29:04.142709 29408 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:29:04.145452 29551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1: Bootstrap starting.
I20260619 12:29:04.145695 29592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:29:04.146632 29592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:29:04.148240 29551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1: Neither blocks nor log segments found. Creating new log.
I20260619 12:29:04.149080 29551 log.cc:824] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1: Log is configured to *not* fsync() on all Append() calls
I20260619 12:29:04.151510 29551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1: No bootstrap required, opened a new log
I20260619 12:29:04.155827 29551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } }
I20260619 12:29:04.156145 29551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 571b0adbe6054726a89c9e9b767c6da1, State: Initialized, Role: FOLLOWER
I20260619 12:29:04.158190 29551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } }
I20260619 12:29:04.161870 29551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:29:04.162088 29525 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cc931d011b184f718a1fb50e9eacc9e3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "571b0adbe6054726a89c9e9b767c6da1"
I20260619 12:29:04.162341 29525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:29:04.164149 29525 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cc931d011b184f718a1fb50e9eacc9e3 in term 1.
I20260619 12:29:04.165235 29598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } } }
I20260619 12:29:04.166623 29598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:04.167562 29403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.142840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56134 (local address 127.6.231.254:46407)
0619 12:29:04.142952 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.142957 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.150514 (+ 7557us) server_negotiation.cc:407] Connection header received
0619 12:29:04.150815 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.150819 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.150874 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.150941 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.152637 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.153496 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.155332 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.155630 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.155706 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.159533 (+ 3827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:04.159538 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:04.159541 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:04.159577 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:04.161917 (+ 2340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.165622 (+ 3705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.165628 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.165630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.165938 (+ 308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.166268 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.166272 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.166273 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.166494 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:04.166624 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:04.167048 (+ 424us) server_negotiation.cc:299] Negotiation successful
0619 12:29:04.167400 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260619 12:29:04.171658 29609 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:29:04.171753 29609 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:29:04.174209 29450 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1 [term 1 FOLLOWER]: Refusing update from remote peer cc931d011b184f718a1fb50e9eacc9e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:29:04.174208 29525 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1 [term 1 FOLLOWER]: Refusing update from remote peer cc931d011b184f718a1fb50e9eacc9e3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:29:04.174971 29558 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:29:04.175810 29558 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:29:04.180377 29573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc931d011b184f718a1fb50e9eacc9e3. Latest consensus state: current_term: 1 leader_uuid: "cc931d011b184f718a1fb50e9eacc9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } } }
I20260619 12:29:04.180495 29573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:04.186060 29598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc931d011b184f718a1fb50e9eacc9e3. Latest consensus state: current_term: 1 leader_uuid: "cc931d011b184f718a1fb50e9eacc9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } } }
I20260619 12:29:04.186209 29598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:04.191721 29555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.150449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42758 (local address 127.6.231.252:35775)
0619 12:29:04.150559 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.150563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.168318 (+ 17755us) server_negotiation.cc:407] Connection header received
0619 12:29:04.168734 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.168740 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.168797 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.168852 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.169724 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.170675 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.172102 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.172449 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.172546 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.187816 (+ 15270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:04.187822 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:04.187825 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:04.187854 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:04.189684 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.190263 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.190267 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.190269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.190356 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.190673 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.190676 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.190678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.190918 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:04.191034 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:04.191399 (+ 365us) server_negotiation.cc:299] Negotiation successful
0619 12:29:04.191528 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260619 12:29:04.194963 29558 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc931d011b184f718a1fb50e9eacc9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } } }
I20260619 12:29:04.195117 29558 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [sys.catalog]: This master's current role is: LEADER
I20260619 12:29:04.195250 29594 mvcc.cc:205] Tried to move back new op lower bound from 7298548302495211520 to 7298548302388920320. Current Snapshot: MvccSnapshot[applied={T|T < 7298548302495211520}]
I20260619 12:29:04.196847 29573 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc931d011b184f718a1fb50e9eacc9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } } }
I20260619 12:29:04.196974 29573 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:04.198652 29558 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cc931d011b184f718a1fb50e9eacc9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } } }
I20260619 12:29:04.198784 29558 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3 [sys.catalog]: This master's current role is: LEADER
I20260619 12:29:04.199499 29598 sys_catalog.cc:455] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cc931d011b184f718a1fb50e9eacc9e3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cc931d011b184f718a1fb50e9eacc9e3" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 46407 } } peers { permanent_uuid: "5167651f0a8042bb89859bee031786c1" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 43905 } } peers { permanent_uuid: "571b0adbe6054726a89c9e9b767c6da1" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 35775 } } }
I20260619 12:29:04.199632 29598 sys_catalog.cc:458] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:04.200260 29592 catalog_manager.cc:1347] Generated new cluster ID: d215cf7af77749a08a042ae08da7616f
I20260619 12:29:04.200323 29592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:29:04.207052 29479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:04.149848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49570 (local address 127.6.231.253:43905)
0619 12:29:04.150018 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.150023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.153679 (+ 3656us) server_negotiation.cc:407] Connection header received
0619 12:29:04.153850 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.153853 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.153901 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.153949 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.154879 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.155641 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.157645 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:04.158034 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:04.158111 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:04.165864 (+ 7753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:04.165868 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:04.165871 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:04.165903 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:04.199668 (+ 33765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.205566 (+ 5898us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.205588 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.205591 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.205716 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:04.206132 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:04.206135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:04.206137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:04.206364 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:04.206499 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:04.206754 (+ 255us) server_negotiation.cc:299] Negotiation successful
0619 12:29:04.206884 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260619 12:29:04.216586 29613 mvcc.cc:205] Tried to move back new op lower bound from 7298548302495211520 to 7298548302388920320. Current Snapshot: MvccSnapshot[applied={T|T < 7298548302495211520}]
I20260619 12:29:04.242898 29592 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:29:04.244258 29592 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:29:04.253116 29592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc931d011b184f718a1fb50e9eacc9e3: Generated new TSK 0
I20260619 12:29:04.254014 29592 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:29:04.295666 29403 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:04.292364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56144 (local address 127.6.231.254:46407)
0619 12:29:04.292476 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.292481 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.294553 (+ 2072us) server_negotiation.cc:407] Connection header received
0619 12:29:04.294758 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.294761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.294814 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.294869 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.295521 (+ 652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56144: BlockingRecv error: recv got EOF from 127.0.0.1:56144 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260619 12:29:04.297271 29479 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:04.291952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49572 (local address 127.6.231.253:43905)
0619 12:29:04.292111 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.292116 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.296354 (+ 4238us) server_negotiation.cc:407] Connection header received
0619 12:29:04.296528 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.296532 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.296584 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.296637 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.297148 (+ 511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49572: BlockingRecv error: recv got EOF from 127.0.0.1:49572 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260619 12:29:04.299022 29555 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:04.293722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42772 (local address 127.6.231.252:35775)
0619 12:29:04.293840 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.293844 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.297914 (+ 4070us) server_negotiation.cc:407] Connection header received
0619 12:29:04.298274 (+ 360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.298278 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.298343 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.298405 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.298888 (+ 483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42772: BlockingRecv error: recv got EOF from 127.0.0.1:42772 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 12:29:04.304124 29479 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:04.302415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49574 (local address 127.6.231.253:43905)
0619 12:29:04.302529 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.302533 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.303077 (+ 544us) server_negotiation.cc:407] Connection header received
0619 12:29:04.303306 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.303310 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.303385 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.303437 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.304001 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49574: BlockingRecv error: recv got EOF from 127.0.0.1:49574 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260619 12:29:04.309844 29479 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:04.307962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49582 (local address 127.6.231.253:43905)
0619 12:29:04.308087 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.308091 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.308670 (+ 579us) server_negotiation.cc:407] Connection header received
0619 12:29:04.308930 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.308934 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.309003 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.309065 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.309699 (+ 634us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49582: BlockingRecv error: recv got EOF from 127.0.0.1:49582 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
W20260619 12:29:04.316828 29479 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:04.314376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49590 (local address 127.6.231.253:43905)
0619 12:29:04.314480 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.314485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.314971 (+ 486us) server_negotiation.cc:407] Connection header received
0619 12:29:04.316080 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.316084 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.316139 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.316192 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.316710 (+ 518us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49590: BlockingRecv error: recv got EOF from 127.0.0.1:49590 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
W20260619 12:29:04.325129 29479 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:04.323348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49604 (local address 127.6.231.253:43905)
0619 12:29:04.323497 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.323502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.324137 (+ 635us) server_negotiation.cc:407] Connection header received
0619 12:29:04.324334 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.324338 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.324391 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.324442 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.325001 (+ 559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49604: BlockingRecv error: recv got EOF from 127.0.0.1:49604 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260619 12:29:04.338068 29479 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:04.336042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49620 (local address 127.6.231.253:43905)
0619 12:29:04.336206 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.336211 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.336839 (+ 628us) server_negotiation.cc:407] Connection header received
0619 12:29:04.337068 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.337072 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.337128 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.337184 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.337925 (+ 741us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49620: BlockingRecv error: recv got EOF from 127.0.0.1:49620 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 12:29:04.358731 29479 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:04.357259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49626 (local address 127.6.231.253:43905)
0619 12:29:04.357397 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.357401 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.357724 (+ 323us) server_negotiation.cc:407] Connection header received
0619 12:29:04.357896 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.357899 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.357965 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.358077 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.358607 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49626: BlockingRecv error: recv got EOF from 127.0.0.1:49626 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260619 12:29:04.394523 29479 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:04.393120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49634 (local address 127.6.231.253:43905)
0619 12:29:04.393261 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.393271 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.393405 (+ 134us) server_negotiation.cc:407] Connection header received
0619 12:29:04.393577 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.393581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.393638 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.393716 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.394375 (+ 659us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49634: BlockingRecv error: recv got EOF from 127.0.0.1:49634 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260619 12:29:04.461812 29479 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:04.460654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49644 (local address 127.6.231.253:43905)
0619 12:29:04.460822 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.460828 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.460846 (+ 18us) server_negotiation.cc:407] Connection header received
0619 12:29:04.461012 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.461015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.461069 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.461159 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.461687 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49644: BlockingRecv error: recv got EOF from 127.0.0.1:49644 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260619 12:29:04.593287 29479 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:04.592088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49660 (local address 127.6.231.253:43905)
0619 12:29:04.592251 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.592256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.592405 (+ 149us) server_negotiation.cc:407] Connection header received
0619 12:29:04.592571 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.592575 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.592628 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.592722 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.593163 (+ 441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49660: BlockingRecv error: recv got EOF from 127.0.0.1:49660 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260619 12:29:04.852901 29479 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:04.851840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49662 (local address 127.6.231.253:43905)
0619 12:29:04.851971 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:04.851975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:04.852100 (+ 125us) server_negotiation.cc:407] Connection header received
0619 12:29:04.852227 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:04.852230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:04.852282 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:04.852379 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:04.852787 (+ 408us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49662: BlockingRecv error: recv got EOF from 127.0.0.1:49662 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260619 12:29:05.138307 29584 catalog_manager.cc:1259] Loaded cluster ID: d215cf7af77749a08a042ae08da7616f
I20260619 12:29:05.138393 29584 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1: loading cluster ID for follower catalog manager: success
I20260619 12:29:05.140013 29584 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1: acquiring CA information for follower catalog manager: success
I20260619 12:29:05.141119 29584 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5167651f0a8042bb89859bee031786c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:29:05.173144 29609 catalog_manager.cc:1259] Loaded cluster ID: d215cf7af77749a08a042ae08da7616f
I20260619 12:29:05.173272 29609 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1: loading cluster ID for follower catalog manager: success
I20260619 12:29:05.175072 29609 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1: acquiring CA information for follower catalog manager: success
I20260619 12:29:05.176314 29609 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 571b0adbe6054726a89c9e9b767c6da1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:29:05.370925 29684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:05.366962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49668 (local address 127.6.231.253:43905)
0619 12:29:05.367282 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:05.367286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:05.367303 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:29:05.367433 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:05.367437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:05.367619 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:05.367738 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:29:05.367997 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:05.368534 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:05.369402 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:05.369695 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:05.369804 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:05.370109 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:29:05.370538 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:29:05.370669 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:05.370712 (+ 43us) server_negotiation.cc:299] Negotiation successful
0619 12:29:05.370764 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":235,"thread_start_us":142,"threads_started":1}
I20260619 12:29:05.377615 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:05.374024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42780 (local address 127.6.231.252:35775)
0619 12:29:05.374266 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:05.374270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:05.374283 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:29:05.374405 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:05.374407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:05.374515 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:05.374651 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:29:05.374817 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:05.375316 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:05.376169 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:05.376396 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:05.376486 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:05.376772 (+ 286us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:29:05.377179 (+ 407us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:29:05.377272 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:05.377332 (+ 60us) server_negotiation.cc:299] Negotiation successful
0619 12:29:05.377380 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":91,"threads_started":1}
I20260619 12:29:05.384888 29684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:05.380878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49682 (local address 127.6.231.253:43905)
0619 12:29:05.381021 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:05.381026 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:05.381246 (+ 220us) server_negotiation.cc:407] Connection header received
0619 12:29:05.381498 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:05.381501 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:05.381558 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:05.381668 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:29:05.381958 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:05.382642 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:05.383791 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:05.384012 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:05.384120 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:05.384429 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:29:05.384510 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:29:05.384583 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:05.384693 (+ 110us) server_negotiation.cc:299] Negotiation successful
0619 12:29:05.384747 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260619 12:29:05.385473 29697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:05.380891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56150 (local address 127.6.231.254:46407)
0619 12:29:05.381160 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:05.381168 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:05.381185 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:29:05.381283 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:05.381288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:05.381505 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:05.381924 (+ 419us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:29:05.382042 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:05.382643 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:05.383771 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:05.384132 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:05.384227 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:05.384538 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:29:05.384907 (+ 369us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:29:05.384985 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:05.385250 (+ 265us) server_negotiation.cc:299] Negotiation successful
0619 12:29:05.385313 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":70,"threads_started":1}
I20260619 12:29:05.385506 29690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:05.381421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42796 (local address 127.6.231.252:35775)
0619 12:29:05.381563 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:05.381567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:05.381695 (+ 128us) server_negotiation.cc:407] Connection header received
0619 12:29:05.381856 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:05.381859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:05.381911 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:05.381973 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:29:05.382280 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:05.383043 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:05.384177 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:05.384501 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:05.384583 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:05.384941 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:29:05.385015 (+ 74us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:29:05.385065 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:05.385328 (+ 263us) server_negotiation.cc:299] Negotiation successful
0619 12:29:05.385376 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260619 12:29:05.388474 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 29333
I20260619 12:29:05.395313 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 29408
I20260619 12:29:05.401953 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 29482
2026-06-19T12:29:05Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3167 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Jun 19 12:29:05 dist-test-slave-kr9h krb5kdc[29701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Jun 19 12:29:05 dist-test-slave-kr9h krb5kdc[29701](info): set up 2 sockets
Jun 19 12:29:05 dist-test-slave-kr9h krb5kdc[29701](info): commencing operation
krb5kdc: starting...
W20260619 12:29:06.454113 7071 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/test-user.keytab.
Jun 19 12:29:06 dist-test-slave-kr9h krb5kdc[29701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-06-19T12:29:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-06-19T12:29:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.6.231.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
Entry for principal kudu/127.6.231.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab.
I20260619 12:29:06.575716 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:42843
--webserver_interface=127.6.231.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:42843,127.6.231.253:35259,127.6.231.252:42271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260619 12:29:06.686685 29714 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:29:06.686981 29714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:29:06.687006 29714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:29:06.687098 29714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:29:06.691146 29714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:29:06.691202 29714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:29:06.691222 29714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:29:06.691239 29714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:29:06.691277 29714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:29:06.696575 29714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:42843,127.6.231.253:35259,127.6.231.252:42271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.254.keytab
--principal=kudu/127.6.231.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.254:42843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.6.231.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:29:06.697721 29714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:29:06.698757 29714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260619 12:29:06.708006 29714 server_base.cc:1034] running on GCE node
W20260619 12:29:06.707945 29722 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:29:06.707964 29720 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:29:06.707929 29719 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:29:06.708791 29714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:29:06.709991 29714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:29:06.711192 29714 hybrid_clock.cc:648] HybridClock initialized: now 1781872146711173 us; error 33 us; skew 500 ppm
Jun 19 12:29:06 dist-test-slave-kr9h krb5kdc[29701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872146, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:29:06.713992 29714 init.cc:377] Logged in from keytab as kudu/127.6.231.254@KRBTEST.COM (short username kudu)
I20260619 12:29:06.714942 29714 webserver.cc:460] Webserver started at http://127.6.231.254:36945/ using document root <none> and password file <none>
I20260619 12:29:06.715499 29714 fs_manager.cc:359] Metadata directory not provided
I20260619 12:29:06.715587 29714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:29:06.715806 29714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:29:06.717916 29714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/data/instance:
uuid: "b45f3fe5e6324e9abbff257591ab9a86"
format_stamp: "Formatted at 2026-06-19 12:29:06 on dist-test-slave-kr9h"
server_key: "8f3e20150bf3536ae970b445311e3e04"
server_key_iv: "8cfbd4d7f64e3b886dbe39715252ae1e"
server_key_version: "encryptionkey@0"
I20260619 12:29:06.718470 29714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/wal/instance:
uuid: "b45f3fe5e6324e9abbff257591ab9a86"
format_stamp: "Formatted at 2026-06-19 12:29:06 on dist-test-slave-kr9h"
server_key: "8f3e20150bf3536ae970b445311e3e04"
server_key_iv: "8cfbd4d7f64e3b886dbe39715252ae1e"
server_key_version: "encryptionkey@0"
I20260619 12:29:06.722214 29714 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:29:06.724678 29729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:29:06.726046 29714 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:29:06.726174 29714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/wal
uuid: "b45f3fe5e6324e9abbff257591ab9a86"
format_stamp: "Formatted at 2026-06-19 12:29:06 on dist-test-slave-kr9h"
server_key: "8f3e20150bf3536ae970b445311e3e04"
server_key_iv: "8cfbd4d7f64e3b886dbe39715252ae1e"
server_key_version: "encryptionkey@0"
I20260619 12:29:06.726279 29714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:29:06.737372 29714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:29:06.740262 29714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:29:06.740454 29714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:29:06.747563 29714 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.254:42843
I20260619 12:29:06.747891 29781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.254:42843 every 8 connection(s)
I20260619 12:29:06.748520 29714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/data/info.pb
I20260619 12:29:06.751431 29782 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:29:06.751987 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 29714
I20260619 12:29:06.752152 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-0/wal/instance
I20260619 12:29:06.752429 7071 external_mini_cluster.cc:1428] Setting key a5140a3f21d97940c35a9e6f1b34142e
I20260619 12:29:06.755528 29782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.231.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.253@KRBTEST.COM" created.
Jun 19 12:29:06 dist-test-slave-kr9h krb5kdc[29701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872146, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:29:06.772156 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:06.757940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:42843 (local address 127.0.0.1:53776)
0619 12:29:06.758440 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:06.758495 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:06.759422 (+ 927us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:06.760340 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:06.760348 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:06.761083 (+ 735us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:06.761398 (+ 315us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:06.761412 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:06.762580 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:06.762582 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:06.763407 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:06.763416 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:06.764319 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:06.764322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:06.764424 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:06.765055 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:06.765071 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:06.766854 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:06.769338 (+ 2484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:06.769346 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:06.769354 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:06.769644 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:06.770009 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:06.770011 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:06.770012 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:06.770121 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:06.770868 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:06.770872 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:06.770989 (+ 117us) client_negotiation.cc:769] Sending connection context
0619 12:29:06.771141 (+ 152us) client_negotiation.cc:240] Negotiation successful
0619 12:29:06.771385 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":134,"threads_started":1}
I20260619 12:29:06.772159 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:06.757941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53776 (local address 127.6.231.254:42843)
0619 12:29:06.758501 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:06.758516 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:06.758624 (+ 108us) server_negotiation.cc:407] Connection header received
0619 12:29:06.759731 (+ 1107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:06.759737 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:06.760004 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:06.760208 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:06.761621 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:06.762466 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:06.763524 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:06.764171 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:06.764360 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:06.767009 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:06.767016 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:06.767032 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:06.767061 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:06.769211 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:06.769763 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:06.769772 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:06.769775 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:06.769919 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:06.770220 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:06.770223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:06.770225 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:06.770761 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:06.770909 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:06.771285 (+ 376us) server_negotiation.cc:299] Negotiation successful
0619 12:29:06.771476 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":434,"spinlock_wait_cycles":11648,"thread_start_us":170,"threads_started":1}
I20260619 12:29:06.774744 29782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:06.775487 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:06.775249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:35259 (local address 127.0.0.1:59442)
0619 12:29:06.775361 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:06.775428 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 12:29:06.775739 29730 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.231.253:35259: connect: Connection refused (error 111)
I20260619 12:29:06.776628 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:06.776408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:35259 (local address 127.0.0.1:59452)
0619 12:29:06.776500 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:06.776581 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260619 12:29:06.776808 29782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.253:35259: Network error: Client connection negotiation failed: client connection to 127.6.231.253:35259: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.253.keytab.
Entry for principal kudu/127.6.231.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.253.keytab.
I20260619 12:29:06.780305 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.253:35259
--webserver_interface=127.6.231.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.253.keytab
--principal=kudu/127.6.231.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:42843,127.6.231.253:35259,127.6.231.252:42271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:29:06.826042 29782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } attempt: 1
I20260619 12:29:06.826956 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:06.826637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:35259 (local address 127.0.0.1:59466)
0619 12:29:06.826787 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:06.826852 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 12:29:06.827764 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:06.827493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:35259 (local address 127.0.0.1:59478)
0619 12:29:06.827595 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:06.827683 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260619 12:29:06.827898 29782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.253:35259: Network error: Client connection negotiation failed: client connection to 127.6.231.253:35259: connect: Connection refused (error 111)
W20260619 12:29:06.893904 29789 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:29:06.894227 29789 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:29:06.894251 29789 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:29:06.894357 29789 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260619 12:29:06.896071 29782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } attempt: 2
I20260619 12:29:06.896953 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:06.896661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:35259 (local address 127.0.0.1:59488)
0619 12:29:06.896815 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:06.896879 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260619 12:29:06.897763 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:06.897533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:35259 (local address 127.0.0.1:59502)
0619 12:29:06.897664 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:06.897710 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.253:35259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260619 12:29:06.897921 29782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.253:35259: Network error: Client connection negotiation failed: client connection to 127.6.231.253:35259: connect: Connection refused (error 111)
W20260619 12:29:06.899185 29789 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:29:06.899240 29789 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:29:06.899259 29789 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:29:06.899277 29789 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:29:06.899329 29789 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:29:06.905032 29789 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:42843,127.6.231.253:35259,127.6.231.252:42271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.253.keytab
--principal=kudu/127.6.231.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.253:35259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.6.231.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29789
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:29:06.906299 29789 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:29:06.907317 29789 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:29:06.915972 29794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:29:06.915956 29797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:29:06.915961 29795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:29:06.916319 29789 server_base.cc:1034] running on GCE node
I20260619 12:29:06.916953 29789 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:29:06.918160 29789 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:29:06.919353 29789 hybrid_clock.cc:648] HybridClock initialized: now 1781872146919325 us; error 50 us; skew 500 ppm
Jun 19 12:29:06 dist-test-slave-kr9h krb5kdc[29701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872146, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:29:06.922406 29789 init.cc:377] Logged in from keytab as kudu/127.6.231.253@KRBTEST.COM (short username kudu)
I20260619 12:29:06.923549 29789 webserver.cc:460] Webserver started at http://127.6.231.253:36997/ using document root <none> and password file <none>
I20260619 12:29:06.924125 29789 fs_manager.cc:359] Metadata directory not provided
I20260619 12:29:06.924192 29789 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:29:06.924466 29789 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:29:06.926786 29789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/data/instance:
uuid: "d79b0179f5e34b80958249c1b7037119"
format_stamp: "Formatted at 2026-06-19 12:29:06 on dist-test-slave-kr9h"
server_key: "8acb72f3b3ccdaa40fd125c6f2d3e074"
server_key_iv: "502a60e3f5db151b1e7bd85fb434c274"
server_key_version: "encryptionkey@0"
I20260619 12:29:06.927402 29789 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/wal/instance:
uuid: "d79b0179f5e34b80958249c1b7037119"
format_stamp: "Formatted at 2026-06-19 12:29:06 on dist-test-slave-kr9h"
server_key: "8acb72f3b3ccdaa40fd125c6f2d3e074"
server_key_iv: "502a60e3f5db151b1e7bd85fb434c274"
server_key_version: "encryptionkey@0"
I20260619 12:29:06.931807 29789 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260619 12:29:06.934594 29804 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:29:06.936276 29789 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260619 12:29:06.936436 29789 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/wal
uuid: "d79b0179f5e34b80958249c1b7037119"
format_stamp: "Formatted at 2026-06-19 12:29:06 on dist-test-slave-kr9h"
server_key: "8acb72f3b3ccdaa40fd125c6f2d3e074"
server_key_iv: "502a60e3f5db151b1e7bd85fb434c274"
server_key_version: "encryptionkey@0"
I20260619 12:29:06.936568 29789 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:29:06.964457 29789 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:29:06.967844 29789 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:29:06.968068 29789 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:29:06.975281 29789 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.253:35259
I20260619 12:29:06.975584 29856 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.253:35259 every 8 connection(s)
I20260619 12:29:06.976316 29789 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/data/info.pb
I20260619 12:29:06.977144 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 29789
I20260619 12:29:06.977283 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-1/wal/instance
I20260619 12:29:06.977645 7071 external_mini_cluster.cc:1428] Setting key a0e158d999e6f08e25fb0fecd8f9ca5e
I20260619 12:29:06.979583 29857 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:29:06.984895 29857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.6.231.252@KRBTEST.COM; defaulting to no policy
I20260619 12:29:06.989120 29782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } attempt: 3
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.6.231.252@KRBTEST.COM" created.
Jun 19 12:29:06 dist-test-slave-kr9h krb5kdc[29701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872146, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.253@KRBTEST.COM
Jun 19 12:29:06 dist-test-slave-kr9h krb5kdc[29701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872146, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.253@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:29:07.001041 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:06.989591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:35259 (local address 127.0.0.1:59518)
0619 12:29:06.989779 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:06.989799 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:06.989873 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:06.990632 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:06.990636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:06.990812 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:06.990930 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:06.990937 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:06.992618 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:06.992624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:06.993177 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:06.993186 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:06.994049 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:06.994053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:06.994151 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:06.994724 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:06.994742 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:06.996438 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:06.998992 (+ 2554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:06.998996 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:06.998998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:06.999266 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:06.999687 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:06.999689 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:06.999690 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:06.999773 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.000508 (+ 735us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:07.000519 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:07.000622 (+ 103us) client_negotiation.cc:769] Sending connection context
0619 12:29:07.000729 (+ 107us) client_negotiation.cc:240] Negotiation successful
0619 12:29:07.000870 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126}
I20260619 12:29:07.002157 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:06.987834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:42843 (local address 127.0.0.1:53788)
0619 12:29:06.988306 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:06.988349 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:06.989208 (+ 859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:06.989865 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:06.989873 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:06.990284 (+ 411us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:06.990993 (+ 709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:06.991014 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:06.991754 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:06.991758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:06.993292 (+ 1534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:06.993305 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:06.993949 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:06.993956 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:06.994121 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:06.994988 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:06.995013 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:06.996753 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:06.999848 (+ 3095us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:06.999858 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:06.999865 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.000169 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.000502 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:07.000504 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:07.000506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.000656 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.001059 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:07.001069 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:07.001215 (+ 146us) client_negotiation.cc:769] Sending connection context
0619 12:29:07.001423 (+ 208us) client_negotiation.cc:240] Negotiation successful
0619 12:29:07.001628 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"mutex_wait_us":272,"spinlock_wait_cycles":7936,"thread_start_us":123,"threads_started":1}
I20260619 12:29:07.002161 29861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:06.989720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59518 (local address 127.6.231.253:35259)
0619 12:29:06.990107 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:06.990115 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:06.990135 (+ 20us) server_negotiation.cc:407] Connection header received
0619 12:29:06.990170 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:06.990173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:06.990419 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:06.990634 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:06.991097 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:06.992508 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:06.993306 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:06.993915 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:06.994109 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:06.996598 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:06.996603 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:06.996614 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:06.996635 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:06.998860 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:06.999406 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:06.999414 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:06.999423 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:06.999578 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:06.999873 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:06.999878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:06.999880 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.000402 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.000545 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.000878 (+ 333us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.001206 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":289,"spinlock_wait_cycles":1280,"thread_start_us":212,"threads_started":1}
I20260619 12:29:07.001770 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:06.987396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53788 (local address 127.6.231.254:42843)
0619 12:29:06.987572 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:06.987577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:06.988460 (+ 883us) server_negotiation.cc:407] Connection header received
0619 12:29:06.989488 (+ 1028us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:06.989491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:06.989558 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:06.989652 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:06.991113 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:06.991647 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:06.993423 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:06.993789 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:06.993894 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:06.997179 (+ 3285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:06.997184 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:06.997187 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:06.997218 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:06.999753 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.000266 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.000271 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.000273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.000347 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.000752 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.000755 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.000757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.000935 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.001044 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.001518 (+ 474us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.001627 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260619 12:29:07.003650 29857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:07.004535 29782 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:07.005183 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.004953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:33940)
0619 12:29:07.005058 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.005113 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260619 12:29:07.005911 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.005641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:33950)
0619 12:29:07.005799 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.005849 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260619 12:29:07.006062 29782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:42271: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.6.231.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.252.keytab.
Entry for principal kudu/127.6.231.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.252.keytab.
Jun 19 12:29:07 dist-test-slave-kr9h krb5kdc[29701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872146, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.253@KRBTEST.COM for kudu/127.6.231.253@KRBTEST.COM
I20260619 12:29:07.008783 7071 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
/tmp/dist-test-taskYaZBJE/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.252:42271
--webserver_interface=127.6.231.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.252.keytab
--principal=kudu/127.6.231.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.6.231.212:33977
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.6.231.254:42843,127.6.231.253:35259,127.6.231.252:42271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260619 12:29:07.012889 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.004168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:35259 (local address 127.0.0.1:59524)
0619 12:29:07.004310 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.004325 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:07.004417 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:07.004622 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:07.004625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:07.004801 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:07.004883 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:07.004888 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.005803 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.005809 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:07.006622 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:07.006631 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.007135 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.007138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:07.007210 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.007776 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:07.007790 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:07.009123 (+ 1333us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:07.011132 (+ 2009us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:07.011139 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:07.011144 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.011431 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.011815 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:07.011818 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:07.011820 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.011923 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.012408 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:07.012411 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:07.012520 (+ 109us) client_negotiation.cc:769] Sending connection context
0619 12:29:07.012624 (+ 104us) client_negotiation.cc:240] Negotiation successful
0619 12:29:07.012754 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260619 12:29:07.013085 29861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.004241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59524 (local address 127.6.231.253:35259)
0619 12:29:07.004321 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.004326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.004376 (+ 50us) server_negotiation.cc:407] Connection header received
0619 12:29:07.004494 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.004497 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.004537 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.004613 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.004981 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.005700 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.006733 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.007042 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.007124 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.009257 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:07.009264 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:07.009271 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:07.009302 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:07.011008 (+ 1706us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.011570 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.011573 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.011574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.011705 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.012015 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.012022 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.012027 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.012290 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.012396 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.012714 (+ 318us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.012859 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260619 12:29:07.014289 29857 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:07.015266 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.014980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:33962)
0619 12:29:07.015114 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.015203 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260619 12:29:07.015492 29805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
I20260619 12:29:07.016448 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.016214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:33968)
0619 12:29:07.016329 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.016390 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260619 12:29:07.016633 29857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:42271: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
I20260619 12:29:07.037231 29782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } attempt: 1
I20260619 12:29:07.038112 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.037824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:33980)
0619 12:29:07.037965 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.038032 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 12:29:07.038874 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.038661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:33988)
0619 12:29:07.038765 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.038818 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260619 12:29:07.039063 29782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:42271: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
I20260619 12:29:07.065930 29857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } attempt: 1
I20260619 12:29:07.066987 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.066629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:33996)
0619 12:29:07.066808 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.066890 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260619 12:29:07.067970 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.067736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:34008)
0619 12:29:07.067850 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.067905 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260619 12:29:07.068145 29857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:42271: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
I20260619 12:29:07.114250 29782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } attempt: 2
I20260619 12:29:07.115120 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.114776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:34024)
0619 12:29:07.114970 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.115041 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260619 12:29:07.115945 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.115732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:34034)
0619 12:29:07.115828 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.115871 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 12:29:07.116171 29782 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:42271: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
W20260619 12:29:07.126271 29863 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260619 12:29:07.126557 29863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260619 12:29:07.126606 29863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260619 12:29:07.126686 29863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260619 12:29:07.131009 29863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260619 12:29:07.131072 29863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260619 12:29:07.131091 29863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260619 12:29:07.131129 29863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260619 12:29:07.131162 29863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260619 12:29:07.136319 29857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } attempt: 2
I20260619 12:29:07.136821 29863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.6.231.212:33977
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.6.231.254:42843,127.6.231.253:35259,127.6.231.252:42271
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/krb5kdc/kudu_127.6.231.252.keytab
--principal=kudu/127.6.231.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.6.231.252:42271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.6.231.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 19 Jun 2026 11:53:36 UTC on ccycloud.quasar-bvtpgp.root.comops.site
build id 4455649
I20260619 12:29:07.137162 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.136835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:34038)
0619 12:29:07.137003 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.137078 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260619 12:29:07.137902 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.137677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:34052)
0619 12:29:07.137786 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.137842 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260619 12:29:07.138036 29857 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.6.231.252:42271: Network error: Client connection negotiation failed: client connection to 127.6.231.252:42271: connect: Connection refused (error 111)
I20260619 12:29:07.138230 29863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260619 12:29:07.139230 29863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260619 12:29:07.147606 29872 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:29:07.147684 29863 server_base.cc:1034] running on GCE node
W20260619 12:29:07.147615 29870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260619 12:29:07.147608 29869 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260619 12:29:07.148411 29863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260619 12:29:07.149787 29863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260619 12:29:07.150997 29863 hybrid_clock.cc:648] HybridClock initialized: now 1781872147150955 us; error 53 us; skew 500 ppm
Jun 19 12:29:07 dist-test-slave-kr9h krb5kdc[29701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872147, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260619 12:29:07.154125 29863 init.cc:377] Logged in from keytab as kudu/127.6.231.252@KRBTEST.COM (short username kudu)
I20260619 12:29:07.154976 29863 webserver.cc:460] Webserver started at http://127.6.231.252:42077/ using document root <none> and password file <none>
I20260619 12:29:07.155510 29863 fs_manager.cc:359] Metadata directory not provided
I20260619 12:29:07.155571 29863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260619 12:29:07.155736 29863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260619 12:29:07.157812 29863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/data/instance:
uuid: "6d35602a06a6455d9c345ac7f4833e5e"
format_stamp: "Formatted at 2026-06-19 12:29:07 on dist-test-slave-kr9h"
server_key: "44c13365fa59231c9de905a2b084ddc2"
server_key_iv: "0b464917c808567a683823d70d5e6800"
server_key_version: "encryptionkey@0"
I20260619 12:29:07.158320 29863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/wal/instance:
uuid: "6d35602a06a6455d9c345ac7f4833e5e"
format_stamp: "Formatted at 2026-06-19 12:29:07 on dist-test-slave-kr9h"
server_key: "44c13365fa59231c9de905a2b084ddc2"
server_key_iv: "0b464917c808567a683823d70d5e6800"
server_key_version: "encryptionkey@0"
I20260619 12:29:07.162060 29863 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260619 12:29:07.164378 29879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260619 12:29:07.165707 29863 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260619 12:29:07.165838 29863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/data,/tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/wal
uuid: "6d35602a06a6455d9c345ac7f4833e5e"
format_stamp: "Formatted at 2026-06-19 12:29:07 on dist-test-slave-kr9h"
server_key: "44c13365fa59231c9de905a2b084ddc2"
server_key_iv: "0b464917c808567a683823d70d5e6800"
server_key_version: "encryptionkey@0"
I20260619 12:29:07.165946 29863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260619 12:29:07.193181 29863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260619 12:29:07.196482 29863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260619 12:29:07.196697 29863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260619 12:29:07.203629 29863 rpc_server.cc:307] RPC server started. Bound to: 127.6.231.252:42271
I20260619 12:29:07.203904 29931 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.6.231.252:42271 every 8 connection(s)
I20260619 12:29:07.204705 29863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/data/info.pb
I20260619 12:29:07.206481 7071 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu as pid 29863
I20260619 12:29:07.206616 7071 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskYaZBJE/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1781871857195028-7071-0/minicluster-data/master-2/wal/instance
I20260619 12:29:07.206936 7071 external_mini_cluster.cc:1428] Setting key 6eeb194fd0730936b7c32f889aaef7e8
I20260619 12:29:07.207962 29932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260619 12:29:07.211935 29932 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:07.215488 29782 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } attempt: 3
Jun 19 12:29:07 dist-test-slave-kr9h krb5kdc[29701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.252@KRBTEST.COM
Jun 19 12:29:07 dist-test-slave-kr9h krb5kdc[29701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872147, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.252@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
Jun 19 12:29:07 dist-test-slave-kr9h krb5kdc[29701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872146, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.254@KRBTEST.COM for kudu/127.6.231.252@KRBTEST.COM
I20260619 12:29:07.223850 29935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.208453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34064 (local address 127.6.231.252:42271)
0619 12:29:07.208907 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.208923 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.208964 (+ 41us) server_negotiation.cc:407] Connection header received
0619 12:29:07.209785 (+ 821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.209804 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.210179 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.210550 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.211551 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.212755 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.214045 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.214770 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.214921 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.217955 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:07.217964 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:07.217977 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:07.218014 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:07.220253 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.220807 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.220812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.220817 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.220929 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.221251 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.221256 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.221258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.221818 (+ 560us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.222045 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.222314 (+ 269us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.222630 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":123,"threads_started":1}
I20260619 12:29:07.225739 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.213790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53792 (local address 127.6.231.254:42843)
0619 12:29:07.213913 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.213917 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.214418 (+ 501us) server_negotiation.cc:407] Connection header received
0619 12:29:07.214551 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.214554 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.214605 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.214678 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.215575 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.216277 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.217741 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.218105 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.218199 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.221614 (+ 3415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:07.221618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:07.221621 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:07.221645 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:07.223405 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.224054 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.224058 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.224060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.224176 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.224574 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.224578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.224580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.224852 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.224955 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.225405 (+ 450us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.225575 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":48}
I20260619 12:29:07.225780 29936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.213760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.254:42843 (local address 127.0.0.1:53792)
0619 12:29:07.214262 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.214310 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:07.214443 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:07.214747 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:07.214752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:07.215227 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:07.215399 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:07.215409 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.216417 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.216424 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:07.217595 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:07.217606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.218220 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.218224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:07.218373 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.218829 (+ 456us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:07.218850 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:07.221491 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:07.223557 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:07.223565 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:07.223571 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.223907 (+ 336us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.224318 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:07.224322 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:07.224325 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.224453 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.224972 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:07.224979 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:07.225129 (+ 150us) client_negotiation.cc:769] Sending connection context
0619 12:29:07.225340 (+ 211us) client_negotiation.cc:240] Negotiation successful
0619 12:29:07.225556 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":49,"thread_start_us":181,"threads_started":1}
I20260619 12:29:07.226383 29785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.215906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:34070)
0619 12:29:07.216075 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.216088 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:07.216151 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:07.216596 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:07.216599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:07.216789 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:07.216877 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:07.216885 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.217953 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.217956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:07.218782 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:07.218793 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.219392 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.219396 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:07.219470 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.219911 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:07.219924 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:07.221331 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:07.224730 (+ 3399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:07.224734 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:07.224737 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.224996 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.225316 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:07.225319 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:07.225321 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.225407 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.225905 (+ 498us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:07.225910 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:07.226018 (+ 108us) client_negotiation.cc:769] Sending connection context
0619 12:29:07.226091 (+ 73us) client_negotiation.cc:240] Negotiation successful
0619 12:29:07.226217 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":5}
I20260619 12:29:07.226925 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.216067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34070 (local address 127.6.231.252:42271)
0619 12:29:07.216372 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.216380 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.216392 (+ 12us) server_negotiation.cc:407] Connection header received
0619 12:29:07.216421 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.216429 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.216460 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.216582 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.217010 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.217867 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.218918 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.219280 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.219391 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.221510 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:07.221513 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:07.221516 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:07.221537 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:07.224626 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.225107 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.225110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.225112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.225195 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.225548 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.225552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.225554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.225775 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.226560 (+ 785us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.226640 (+ 80us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.226757 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":227,"thread_start_us":135,"threads_started":1}
I20260619 12:29:07.227267 29932 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:07.229208 29857 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } attempt: 3
I20260619 12:29:07.230969 29782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86: Bootstrap starting.
Jun 19 12:29:07 dist-test-slave-kr9h krb5kdc[29701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872147, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.252@KRBTEST.COM for kudu/127.6.231.253@KRBTEST.COM
Jun 19 12:29:07 dist-test-slave-kr9h krb5kdc[29701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.254@KRBTEST.COM
I20260619 12:29:07.237023 29782 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86: Neither blocks nor log segments found. Creating new log.
I20260619 12:29:07.238034 29782 log.cc:824] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86: Log is configured to *not* fsync() on all Append() calls
I20260619 12:29:07.241886 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.227091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53794 (local address 127.6.231.254:42843)
0619 12:29:07.227376 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.227380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.227724 (+ 344us) server_negotiation.cc:407] Connection header received
0619 12:29:07.227903 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.227906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.227955 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.228033 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.228760 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.230015 (+ 1255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.231255 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.231557 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.231662 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.235794 (+ 4132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:07.235801 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:07.235807 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:07.235844 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:07.238337 (+ 2493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.239526 (+ 1189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.239530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.239532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.239654 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.240103 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.240107 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.240109 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.240403 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.241100 (+ 697us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.241588 (+ 488us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.241727 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209,"thread_start_us":99,"threads_started":1}
I20260619 12:29:07.242869 29782 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86: No bootstrap required, opened a new log
I20260619 12:29:07.243256 29861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.227824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59538 (local address 127.6.231.253:35259)
0619 12:29:07.227949 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.227954 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.228107 (+ 153us) server_negotiation.cc:407] Connection header received
0619 12:29:07.228311 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.228315 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.228382 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.228481 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.228911 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.229645 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.230586 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.230865 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.230962 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.234883 (+ 3921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:07.234910 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:07.234913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:07.234939 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:07.238092 (+ 3153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.239345 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.239348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.239350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.239492 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.240012 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.240015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.240017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.241000 (+ 983us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.241131 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.241697 (+ 566us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.242972 (+ 1275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Jun 19 12:29:07 dist-test-slave-kr9h krb5kdc[29701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872146, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.253@KRBTEST.COM for kudu/127.6.231.252@KRBTEST.COM
I20260619 12:29:07.244436 29936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.227722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.253:35259 (local address 127.0.0.1:59538)
0619 12:29:07.228042 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.228055 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:07.228171 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:07.228516 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:07.228519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:07.228677 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:07.228795 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:07.228802 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.229814 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.229818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:07.230474 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:07.230483 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.231142 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.231145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:07.231248 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.231979 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:07.231991 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:07.234620 (+ 2629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:07.238934 (+ 4314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:07.238940 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:07.238942 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.239225 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.239669 (+ 444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:07.239672 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:07.239674 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.239790 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.241238 (+ 1448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:07.241241 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:07.241387 (+ 146us) client_negotiation.cc:769] Sending connection context
0619 12:29:07.243948 (+ 2561us) client_negotiation.cc:240] Negotiation successful
0619 12:29:07.244159 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256}
I20260619 12:29:07.245736 29932 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } has no permanent_uuid. Determining permanent_uuid...
I20260619 12:29:07.247690 29782 raft_consensus.cc:348] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } }
I20260619 12:29:07.247931 29782 raft_consensus.cc:729] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45f3fe5e6324e9abbff257591ab9a86, State: Initialized, Role: FOLLOWER
I20260619 12:29:07.248296 29782 consensus_queue.cc:260] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } }
I20260619 12:29:07.248510 29860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.229621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:34076)
0619 12:29:07.229756 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.229770 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:07.229835 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:07.234187 (+ 4352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:07.234190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:07.234550 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:07.234702 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:07.234710 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.238678 (+ 3968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.238682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:07.239540 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:07.239551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.240315 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.240318 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:07.240402 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.241890 (+ 1488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:07.241904 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:07.244141 (+ 2237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:07.246964 (+ 2823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:07.246968 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:07.246970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.247233 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.247517 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:07.247520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:07.247522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.247614 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.248041 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:07.248044 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:07.248137 (+ 93us) client_negotiation.cc:769] Sending connection context
0619 12:29:07.248230 (+ 93us) client_negotiation.cc:240] Negotiation successful
0619 12:29:07.248350 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260619 12:29:07.249080 29937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.233680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34076 (local address 127.6.231.252:42271)
0619 12:29:07.233830 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.233834 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.233848 (+ 14us) server_negotiation.cc:407] Connection header received
0619 12:29:07.233910 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.233913 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.233987 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.235275 (+ 1288us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.235442 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.238379 (+ 2937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.239663 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.240104 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.240201 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.244343 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:07.244347 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:07.244350 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:07.244411 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:07.246854 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.247336 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.247340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.247342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.247417 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.247711 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.247715 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.247718 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.247896 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.247972 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.248803 (+ 831us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.248923 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:29:07.249975 29782 sys_catalog.cc:564] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [sys.catalog]: configured and running, proceeding with master startup.
Jun 19 12:29:07 dist-test-slave-kr9h krb5kdc[29701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872146, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.6.231.253@KRBTEST.COM
I20260619 12:29:07.252348 29857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119: Bootstrap starting.
I20260619 12:29:07.252707 29941 sys_catalog.cc:455] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } } }
I20260619 12:29:07.254983 29941 sys_catalog.cc:458] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:07.255156 29861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.243627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59550 (local address 127.6.231.253:35259)
0619 12:29:07.243892 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.243896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.244022 (+ 126us) server_negotiation.cc:407] Connection header received
0619 12:29:07.244191 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.244194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.244234 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.244289 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.245530 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.246602 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.247663 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.247972 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.248052 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.250967 (+ 2915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:07.250971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:07.250974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:07.251003 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:07.252676 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.253288 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.253295 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.253299 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.253388 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.253730 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.253734 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.253735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.253958 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.254137 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.254850 (+ 713us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.254979 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":130,"threads_started":1}
W20260619 12:29:07.255333 29952 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:29:07.255409 29952 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:29:07.255885 29857 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119: Neither blocks nor log segments found. Creating new log.
I20260619 12:29:07.256764 29857 log.cc:824] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119: Log is configured to *not* fsync() on all Append() calls
I20260619 12:29:07.259044 29857 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119: No bootstrap required, opened a new log
I20260619 12:29:07.255631 29714 master_runner.cc:186] Error getting master registration for 127.6.231.253:35259: OK, instance_id { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" instance_seqno: 1781872146973131 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:29:07.259541 29714 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Jun 19 12:29:07 dist-test-slave-kr9h krb5kdc[29701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1781872147, etypes {rep=17 tkt=17 ses=17}, kudu/127.6.231.252@KRBTEST.COM for kudu/127.6.231.252@KRBTEST.COM
I20260619 12:29:07.260040 7071 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260619 12:29:07.261265 29756 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b45f3fe5e6324e9abbff257591ab9a86"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53794
I20260619 12:29:07.261531 29756 raft_consensus.cc:482] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260619 12:29:07.261672 29756 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:29:07.262825 29857 raft_consensus.cc:348] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } }
I20260619 12:29:07.263041 29857 raft_consensus.cc:729] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d79b0179f5e34b80958249c1b7037119, State: Initialized, Role: FOLLOWER
I20260619 12:29:07.263146 29756 raft_consensus.cc:504] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } }
I20260619 12:29:07.263399 29857 consensus_queue.cc:260] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } }
I20260619 12:29:07.263473 29936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.246244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.6.231.252:42271 (local address 127.0.0.1:34082)
0619 12:29:07.246330 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0619 12:29:07.246343 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0619 12:29:07.246413 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0619 12:29:07.247291 (+ 878us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0619 12:29:07.247295 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0619 12:29:07.247457 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0619 12:29:07.247564 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:07.247573 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.248928 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.248930 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:07.257471 (+ 8541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0619 12:29:07.257486 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.258149 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.258151 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0619 12:29:07.258201 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.258664 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0619 12:29:07.258677 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0619 12:29:07.260041 (+ 1364us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0619 12:29:07.261982 (+ 1941us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:07.261985 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:07.261987 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.262211 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.262462 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0619 12:29:07.262464 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0619 12:29:07.262466 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0619 12:29:07.262545 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0619 12:29:07.262961 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0619 12:29:07.262966 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0619 12:29:07.263068 (+ 102us) client_negotiation.cc:769] Sending connection context
0619 12:29:07.263177 (+ 109us) client_negotiation.cc:240] Negotiation successful
0619 12:29:07.263318 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25}
I20260619 12:29:07.264012 29935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.246383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34082 (local address 127.6.231.252:42271)
0619 12:29:07.247027 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.247033 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.247043 (+ 10us) server_negotiation.cc:407] Connection header received
0619 12:29:07.247077 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.247080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.247121 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.247197 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.247677 (+ 480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.248826 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.257618 (+ 8792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.257953 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.258048 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.260196 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:07.260200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:07.260202 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:07.260227 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:07.261869 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.262308 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.262311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.262312 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.262377 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.262628 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.262631 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.262632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.262833 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.262924 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.263767 (+ 843us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.263876 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573}
I20260619 12:29:07.264148 29756 leader_election.cc:288] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [CANDIDATE]: Term 1 election: Requested vote from peers d79b0179f5e34b80958249c1b7037119 (127.6.231.253:35259), 6d35602a06a6455d9c345ac7f4833e5e (127.6.231.252:42271)
I20260619 12:29:07.264817 29857 sys_catalog.cc:564] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119 [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:29:07.264860 29831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b45f3fe5e6324e9abbff257591ab9a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d79b0179f5e34b80958249c1b7037119"
I20260619 12:29:07.265082 29831 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119 [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:29:07.266378 29831 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45f3fe5e6324e9abbff257591ab9a86 in term 1.
I20260619 12:29:07.266809 29730 leader_election.cc:302] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b45f3fe5e6324e9abbff257591ab9a86, d79b0179f5e34b80958249c1b7037119; no voters:
I20260619 12:29:07.267104 29941 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [term 1 FOLLOWER]: Leader election won for term 1
I20260619 12:29:07.267139 29932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e: Bootstrap starting.
I20260619 12:29:07.267303 29941 raft_consensus.cc:686] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [term 1 LEADER]: Becoming Leader. State: Replica: b45f3fe5e6324e9abbff257591ab9a86, State: Running, Role: LEADER
I20260619 12:29:07.267637 29941 consensus_queue.cc:237] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } }
I20260619 12:29:07.266875 29955 sys_catalog.cc:455] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } } }
I20260619 12:29:07.268322 29955 sys_catalog.cc:458] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:07.269332 29941 sys_catalog.cc:455] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b45f3fe5e6324e9abbff257591ab9a86. Latest consensus state: current_term: 1 leader_uuid: "b45f3fe5e6324e9abbff257591ab9a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } } }
I20260619 12:29:07.269483 29941 sys_catalog.cc:458] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [sys.catalog]: This master's current role is: LEADER
I20260619 12:29:07.270370 29932 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e: Neither blocks nor log segments found. Creating new log.
W20260619 12:29:07.270393 29970 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:29:07.270462 29970 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:29:07.271260 29932 log.cc:824] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e: Log is configured to *not* fsync() on all Append() calls
I20260619 12:29:07.275110 29932 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e: No bootstrap required, opened a new log
I20260619 12:29:07.273952 29789 master_runner.cc:186] Error getting master registration for 127.6.231.252:42271: OK, instance_id { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" instance_seqno: 1781872147201593 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260619 12:29:07.277362 29789 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260619 12:29:07.279495 29932 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } }
I20260619 12:29:07.279743 29932 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d35602a06a6455d9c345ac7f4833e5e, State: Initialized, Role: FOLLOWER
I20260619 12:29:07.280104 29932 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } }
I20260619 12:29:07.280706 29976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } } }
I20260619 12:29:07.280936 29976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:07.281601 29932 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e [sys.catalog]: configured and running, proceeding with master startup.
I20260619 12:29:07.281913 29906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b45f3fe5e6324e9abbff257591ab9a86" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6d35602a06a6455d9c345ac7f4833e5e"
I20260619 12:29:07.282141 29906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e [term 0 FOLLOWER]: Advancing to term 1
I20260619 12:29:07.283674 29906 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45f3fe5e6324e9abbff257591ab9a86 in term 1.
I20260619 12:29:07.287477 29974 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260619 12:29:07.290887 29974 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260619 12:29:07.296389 29861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.281935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59558 (local address 127.6.231.253:35259)
0619 12:29:07.282068 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.282073 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.284105 (+ 2032us) server_negotiation.cc:407] Connection header received
0619 12:29:07.284306 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.284309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.284379 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.284437 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.285562 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.286315 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.288240 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.288578 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.288650 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.292301 (+ 3651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:07.292306 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:07.292309 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:07.292340 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:07.294284 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.294897 (+ 613us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.294903 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.294907 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.295012 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.295347 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.295353 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.295354 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.295600 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.295703 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.296070 (+ 367us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.296200 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260619 12:29:07.296721 29935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.282698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34088 (local address 127.6.231.252:42271)
0619 12:29:07.282807 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.282811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.285699 (+ 2888us) server_negotiation.cc:407] Connection header received
0619 12:29:07.285817 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.285821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.285866 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.285941 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.286851 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.287565 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.288617 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.288929 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.289016 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.291160 (+ 2144us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:07.291163 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:07.291166 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:07.291190 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:07.294864 (+ 3674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.295370 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.295374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.295375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.295463 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.295766 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.295769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.295771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.295983 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.296078 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.296424 (+ 346us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.296558 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260619 12:29:07.300892 29990 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260619 12:29:07.300985 29990 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260619 12:29:07.309734 29786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:07.283432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53804 (local address 127.6.231.254:42843)
0619 12:29:07.283565 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.283569 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.293743 (+ 10174us) server_negotiation.cc:407] Connection header received
0619 12:29:07.293859 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.293862 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.293967 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.294152 (+ 185us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.295397 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.296197 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.299117 (+ 2920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:07.299478 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:07.299561 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:07.305812 (+ 6251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0619 12:29:07.305816 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0619 12:29:07.305820 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0619 12:29:07.305852 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0619 12:29:07.307714 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.308257 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.308261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.308263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.308347 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0619 12:29:07.308666 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0619 12:29:07.308672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0619 12:29:07.308673 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0619 12:29:07.308906 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0619 12:29:07.309020 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:07.309377 (+ 357us) server_negotiation.cc:299] Negotiation successful
0619 12:29:07.309550 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260619 12:29:07.314407 29906 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e [term 1 FOLLOWER]: Refusing update from remote peer b45f3fe5e6324e9abbff257591ab9a86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:29:07.314497 29831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119 [term 1 FOLLOWER]: Refusing update from remote peer b45f3fe5e6324e9abbff257591ab9a86: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260619 12:29:07.315204 29941 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:29:07.315383 29971 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260619 12:29:07.320216 29976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b45f3fe5e6324e9abbff257591ab9a86. Latest consensus state: current_term: 1 leader_uuid: "b45f3fe5e6324e9abbff257591ab9a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } } }
I20260619 12:29:07.320353 29976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:07.326478 29955 sys_catalog.cc:455] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b45f3fe5e6324e9abbff257591ab9a86. Latest consensus state: current_term: 1 leader_uuid: "b45f3fe5e6324e9abbff257591ab9a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } } }
I20260619 12:29:07.326638 29955 sys_catalog.cc:458] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:07.336901 29941 sys_catalog.cc:455] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b45f3fe5e6324e9abbff257591ab9a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } } }
I20260619 12:29:07.337069 29941 sys_catalog.cc:458] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [sys.catalog]: This master's current role is: LEADER
I20260619 12:29:07.337203 29991 mvcc.cc:205] Tried to move back new op lower bound from 7298548315391156224 to 7298548315211718656. Current Snapshot: MvccSnapshot[applied={T|T < 7298548315391156224}]
I20260619 12:29:07.338413 29955 sys_catalog.cc:455] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b45f3fe5e6324e9abbff257591ab9a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } } }
I20260619 12:29:07.338567 29955 sys_catalog.cc:458] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119 [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:07.340436 29971 sys_catalog.cc:455] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b45f3fe5e6324e9abbff257591ab9a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } } }
I20260619 12:29:07.340584 29971 sys_catalog.cc:458] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86 [sys.catalog]: This master's current role is: LEADER
I20260619 12:29:07.341745 29976 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b45f3fe5e6324e9abbff257591ab9a86" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b45f3fe5e6324e9abbff257591ab9a86" member_type: VOTER last_known_addr { host: "127.6.231.254" port: 42843 } } peers { permanent_uuid: "d79b0179f5e34b80958249c1b7037119" member_type: VOTER last_known_addr { host: "127.6.231.253" port: 35259 } } peers { permanent_uuid: "6d35602a06a6455d9c345ac7f4833e5e" member_type: VOTER last_known_addr { host: "127.6.231.252" port: 42271 } } }
I20260619 12:29:07.341895 29976 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e [sys.catalog]: This master's current role is: FOLLOWER
I20260619 12:29:07.343353 29974 catalog_manager.cc:1347] Generated new cluster ID: cd83a543d07a4f1b838ca5e0e95e7152
I20260619 12:29:07.343428 29974 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260619 12:29:07.346072 29993 mvcc.cc:205] Tried to move back new op lower bound from 7298548315391156224 to 7298548315211718656. Current Snapshot: MvccSnapshot[applied={T|T < 7298548315391156224}]
I20260619 12:29:07.346451 29995 mvcc.cc:205] Tried to move back new op lower bound from 7298548315391156224 to 7298548315211718656. Current Snapshot: MvccSnapshot[applied={T|T < 7298548315391156224}]
I20260619 12:29:07.379979 29974 catalog_manager.cc:1370] Generated new certificate authority record
I20260619 12:29:07.381451 29974 catalog_manager.cc:1504] Loading token signing keys...
I20260619 12:29:07.401954 29974 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b45f3fe5e6324e9abbff257591ab9a86: Generated new TSK 0
I20260619 12:29:07.402963 29974 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260619 12:29:07.439731 29786 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:07.436844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53806 (local address 127.6.231.254:42843)
0619 12:29:07.436981 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.436985 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.438524 (+ 1539us) server_negotiation.cc:407] Connection header received
0619 12:29:07.438749 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.438752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.438809 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.438867 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.439591 (+ 724us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53806: BlockingRecv error: recv got EOF from 127.0.0.1:53806 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260619 12:29:07.441334 29861 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:07.436439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59566 (local address 127.6.231.253:35259)
0619 12:29:07.436617 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.436624 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.440287 (+ 3663us) server_negotiation.cc:407] Connection header received
0619 12:29:07.440477 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.440481 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.440536 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.440592 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.441208 (+ 616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59566: BlockingRecv error: recv got EOF from 127.0.0.1:59566 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260619 12:29:07.442924 29935 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:07.437941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34096 (local address 127.6.231.252:42271)
0619 12:29:07.438075 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.438080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.441906 (+ 3826us) server_negotiation.cc:407] Connection header received
0619 12:29:07.442088 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.442091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.442152 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.442218 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.442806 (+ 588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34096: BlockingRecv error: recv got EOF from 127.0.0.1:34096 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260619 12:29:07.448019 29861 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:07.446138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59578 (local address 127.6.231.253:35259)
0619 12:29:07.446258 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.446262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.446819 (+ 557us) server_negotiation.cc:407] Connection header received
0619 12:29:07.447052 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.447056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.447137 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.447194 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.447876 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59578: BlockingRecv error: recv got EOF from 127.0.0.1:59578 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 12:29:07.455549 29861 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:07.453825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59582 (local address 127.6.231.253:35259)
0619 12:29:07.453954 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.453960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.454522 (+ 562us) server_negotiation.cc:407] Connection header received
0619 12:29:07.454743 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.454747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.454801 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.454857 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.455411 (+ 554us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59582: BlockingRecv error: recv got EOF from 127.0.0.1:59582 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260619 12:29:07.462373 29861 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:07.460994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59586 (local address 127.6.231.253:35259)
0619 12:29:07.461103 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.461107 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.461336 (+ 229us) server_negotiation.cc:407] Connection header received
0619 12:29:07.461581 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.461585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.461639 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.461694 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.462251 (+ 557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59586: BlockingRecv error: recv got EOF from 127.0.0.1:59586 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260619 12:29:07.471050 29861 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:07.469169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59602 (local address 127.6.231.253:35259)
0619 12:29:07.469331 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.469336 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.469955 (+ 619us) server_negotiation.cc:407] Connection header received
0619 12:29:07.470186 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.470190 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.470249 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.470310 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.470928 (+ 618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59602: BlockingRecv error: recv got EOF from 127.0.0.1:59602 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
W20260619 12:29:07.483419 29861 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:07.481807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59618 (local address 127.6.231.253:35259)
0619 12:29:07.481927 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.481932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.482456 (+ 524us) server_negotiation.cc:407] Connection header received
0619 12:29:07.482650 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.482654 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.482706 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.482758 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.483291 (+ 533us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59618: BlockingRecv error: recv got EOF from 127.0.0.1:59618 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260619 12:29:07.503825 29861 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:07.502094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59634 (local address 127.6.231.253:35259)
0619 12:29:07.502215 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.502220 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.502782 (+ 562us) server_negotiation.cc:407] Connection header received
0619 12:29:07.502998 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.503002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.503057 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.503113 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.503693 (+ 580us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59634: BlockingRecv error: recv got EOF from 127.0.0.1:59634 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260619 12:29:07.539994 29861 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:07.538664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59648 (local address 127.6.231.253:35259)
0619 12:29:07.538811 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.538818 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.539076 (+ 258us) server_negotiation.cc:407] Connection header received
0619 12:29:07.539182 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.539185 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.539238 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.539320 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.539848 (+ 528us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59648: BlockingRecv error: recv got EOF from 127.0.0.1:59648 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260619 12:29:07.607419 29861 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:07.606242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59658 (local address 127.6.231.253:35259)
0619 12:29:07.606440 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.606444 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.606508 (+ 64us) server_negotiation.cc:407] Connection header received
0619 12:29:07.606705 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.606708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.606761 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.606838 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.607295 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59658: BlockingRecv error: recv got EOF from 127.0.0.1:59658 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
W20260619 12:29:07.738811 29861 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:07.737637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59672 (local address 127.6.231.253:35259)
0619 12:29:07.737795 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.737802 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.737959 (+ 157us) server_negotiation.cc:407] Connection header received
0619 12:29:07.738045 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.738048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.738109 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.738189 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.738668 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59672: BlockingRecv error: recv got EOF from 127.0.0.1:59672 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260619 12:29:07.998114 29861 negotiation.cc:337] Failed RPC negotiation. Trace:
0619 12:29:07.996958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59680 (local address 127.6.231.253:35259)
0619 12:29:07.997162 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:07.997166 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:07.997183 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:29:07.997301 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:07.997304 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:07.997353 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:07.997555 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0619 12:29:07.997982 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59680: BlockingRecv error: recv got EOF from 127.0.0.1:59680 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
I20260619 12:29:08.271775 29970 catalog_manager.cc:1259] Loaded cluster ID: cd83a543d07a4f1b838ca5e0e95e7152
I20260619 12:29:08.271867 29970 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119: loading cluster ID for follower catalog manager: success
I20260619 12:29:08.273610 29970 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119: acquiring CA information for follower catalog manager: success
I20260619 12:29:08.274792 29970 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d79b0179f5e34b80958249c1b7037119: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:29:08.302404 29990 catalog_manager.cc:1259] Loaded cluster ID: cd83a543d07a4f1b838ca5e0e95e7152
I20260619 12:29:08.302505 29990 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e: loading cluster ID for follower catalog manager: success
I20260619 12:29:08.304121 29990 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e: acquiring CA information for follower catalog manager: success
I20260619 12:29:08.305261 29990 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6d35602a06a6455d9c345ac7f4833e5e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260619 12:29:08.516769 30064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:08.512337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59694 (local address 127.6.231.253:35259)
0619 12:29:08.512617 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:08.512623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:08.512643 (+ 20us) server_negotiation.cc:407] Connection header received
0619 12:29:08.512727 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:08.512731 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:08.512878 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:08.513006 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:29:08.513300 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:08.513982 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:08.515090 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:08.515354 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:08.515469 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:08.515819 (+ 350us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:29:08.516306 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:29:08.516458 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:08.516527 (+ 69us) server_negotiation.cc:299] Negotiation successful
0619 12:29:08.516593 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":71,"threads_started":1}
I20260619 12:29:08.523983 30070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:08.520124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34110 (local address 127.6.231.252:42271)
0619 12:29:08.520444 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:08.520449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:08.520466 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:29:08.520564 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:08.520566 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:08.520690 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:08.520806 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:29:08.521065 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:08.521665 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:08.522516 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:08.522734 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:08.522835 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:08.523158 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:29:08.523605 (+ 447us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:29:08.523739 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:08.523773 (+ 34us) server_negotiation.cc:299] Negotiation successful
0619 12:29:08.523826 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":66,"threads_started":1}
I20260619 12:29:08.531570 30064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:08.527047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59698 (local address 127.6.231.253:35259)
0619 12:29:08.527329 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:08.527334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:08.527351 (+ 17us) server_negotiation.cc:407] Connection header received
0619 12:29:08.527736 (+ 385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:08.527743 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:08.527830 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:08.527931 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:29:08.528386 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:08.528972 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:08.530204 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:08.530603 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:08.530708 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:08.530968 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:29:08.531078 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:29:08.531153 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:08.531325 (+ 172us) server_negotiation.cc:299] Negotiation successful
0619 12:29:08.531400 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175}
I20260619 12:29:08.531745 30070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:08.527599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34112 (local address 127.6.231.252:42271)
0619 12:29:08.527728 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:08.527733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:08.527746 (+ 13us) server_negotiation.cc:407] Connection header received
0619 12:29:08.527883 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:08.527890 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:08.527948 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:08.528026 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:29:08.528350 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:08.529250 (+ 900us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:08.530514 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:08.530840 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:08.530936 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:08.531288 (+ 352us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:29:08.531365 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:29:08.531428 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:08.531566 (+ 138us) server_negotiation.cc:299] Negotiation successful
0619 12:29:08.531621 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260619 12:29:08.532927 30077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0619 12:29:08.527148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53820 (local address 127.6.231.254:42843)
0619 12:29:08.527399 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0619 12:29:08.527404 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0619 12:29:08.527690 (+ 286us) server_negotiation.cc:407] Connection header received
0619 12:29:08.527961 (+ 271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0619 12:29:08.527963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0619 12:29:08.528151 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0619 12:29:08.528271 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0619 12:29:08.528619 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:08.529549 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:08.530608 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0619 12:29:08.530987 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0619 12:29:08.531083 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0619 12:29:08.531532 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0619 12:29:08.532171 (+ 639us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0619 12:29:08.532242 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0619 12:29:08.532710 (+ 468us) server_negotiation.cc:299] Negotiation successful
0619 12:29:08.532781 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":74,"threads_started":1}
I20260619 12:29:08.536340 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 29714
I20260619 12:29:08.543674 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 29789
I20260619 12:29:08.550307 7071 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskYaZBJE/build/debug/bin/kudu with pid 29863
2026-06-19T12:29:08Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3149 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6316 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (291350 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST